-- Logs begin at Wed 2022-08-31 03:26:41 UTC, end at Wed 2022-08-31 04:29:05 UTC. -- Aug 31 03:52:21.165492 dv-cinder-853910-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 03:52:21.175241 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 03:52:21 2022] *** Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: nodename: dv-cinder-853910-5-iscsi Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: machine: x86_64 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: clock source: unix Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: pcre jit disabled Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: detected number of CPU cores: 4 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: current working directory: / Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: detected binary path: /usr/bin/uwsgi-core Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: your processes number limit is 47680 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: your memory page size is 4096 bytes Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: detected max file descriptor number: 2048 Aug 31 03:52:21.177288 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: lock engine: pthread robust mutexes Aug 31 03:52:21.177981 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: thunder lock: enabled Aug 31 03:52:21.177981 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 03:52:21.177981 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 03:52:21.216354 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: Python main interpreter initialized at 0x56108843d9c0 Aug 31 03:52:21.216354 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: python threads support enabled Aug 31 03:52:21.216354 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: your server socket listen backlog is limited to 100 connections Aug 31 03:52:21.216354 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: your mercy for graceful operations on workers is 90 seconds Aug 31 03:52:21.216773 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 03:52:21.216773 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: *** Operational MODE: preforking *** Aug 31 03:52:21.217030 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: *** uWSGI is running in multiple interpreter mode *** Aug 31 03:52:21.217030 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: spawned uWSGI master process (pid: 81445) Aug 31 03:52:21.217239 dv-cinder-853910-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 03:52:21.223085 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 1 (pid: 81459, cores: 1) Aug 31 03:52:21.223085 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 2 (pid: 81460, cores: 1) Aug 31 03:52:21.223085 dv-cinder-853910-5-iscsi devstack@keystone.service[81445]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 03:52:25.462900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 03:52:25.462900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Aug 31 03:52:25.463756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 03:52:25.463756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Aug 31 03:52:25.463874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 03:52:25.463874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: from cryptography.utils import int_from_bytes Aug 31 03:52:25.464295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 03:52:25.464295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: from cryptography.utils import int_from_bytes Aug 31 03:52:25.513359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 03:52:25.513561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 03:52:25.513640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 03:52:25.513715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 03:52:25.513781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] command line args: [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 03:52:25.513846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] command line args: [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 03:52:25.513986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 03:52:25.514130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 03:52:25.514211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ================================================================================ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 03:52:25.514288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ================================================================================ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 03:52:25.514364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] admin_token = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_dir = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_dir = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.514968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_source = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_source = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] debug = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] debug = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.515932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_config_append = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_config_append = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.516994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_dir = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_dir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.517979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.518990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.519933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.520731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] publish_errors = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] publish_errors = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.521897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] standard_threads = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] standard_threads = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.522904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_journal = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_journal = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_json = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_json = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_stderr = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_stderr = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_syslog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.523987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_syslog = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.524059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.524135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 03:52:25.524330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.524433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.524569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.524651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.524823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.530531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.530626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.530755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.530852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.530929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.531962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.532931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.533986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.534993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.535983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.536975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.537986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.538925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.539948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.540940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.541998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.542987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.543938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.544922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.545894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.546932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.547887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.external = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.548966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.external = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.password = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.token = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.549913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.password = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.token = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.550842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.551960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.552988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.553889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.554903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.555910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.556922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.557954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.558883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.559827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.560792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.561962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.562920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.563972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.564965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.565997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.566879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.567867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.568951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.569904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.570989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.571968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.572971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.573902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.574847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.575948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.576983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.577909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.578901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.579899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.580983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.581999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.582926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.583901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.584917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.585975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.586952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.587914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.588840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.589946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.590967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.591909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.592878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.593989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.594929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.595966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.596982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.597930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.598856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.599003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.599159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.599232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 03:52:25.599305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.599544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.599774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.600015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.600282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.600546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.600789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.601035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.601273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.601513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.601749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.601984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.602217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.602451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.602693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.602932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.603194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.603431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.603678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.603911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.604155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.604385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.604623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.604857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 03:52:25.605024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 03:52:25.629088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.629314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.629538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.630362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.630594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.630816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.637878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.639157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.666032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.666267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.667130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.667368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.719347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.719574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.767850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.768021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.768498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.768742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.923897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.924987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.925109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.925350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.947847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.947983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.948960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.949097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.949221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.949383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.949517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.960199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.960582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.973922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.974069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.974195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.974276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.974350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.974436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 03:52:25.991067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.991197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.991280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 03:52:25.991355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 03:52:25.991611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.991757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.991842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 03:52:25.991918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 03:52:25.992055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.992933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:25.993063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.002850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.002979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.003797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.010581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.010706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.013522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.013645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.013796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.013921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.014837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.015939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.016030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.016257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.016463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.038833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.038957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.040297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.040421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.040566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.040728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.045149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.045336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.045428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.045562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.045718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.045848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.052104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.052231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.052384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.052510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.056432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.056558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.056723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.056849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.057900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.058165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.058432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.059605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.059894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.060164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.075422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.075665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.075776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.076086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.079579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.080116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.082760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.083019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.083175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.083504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.087236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.087803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.090319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.090907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.093521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.093816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.093978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.094925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.095184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.095313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.095617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.110957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.111272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.111395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.111545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.111672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.111760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.111977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.112250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.118553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.118937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.120107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.120615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.121937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.122221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.122449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.122759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.124858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.125143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.125430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.125635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.125820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.125995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.126187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.126314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.126488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.126763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.137246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.137528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.137901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.138179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.138995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.139139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.139370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.139655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.154415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.155188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.157380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.158453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.160227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.161341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.161641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.162783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.163242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.163523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.163797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.164293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.164871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.170177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.171480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.171661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.173055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.174831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.175105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.175450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.175750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.175928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.176250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.176534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.176825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.183715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.184003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.184279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.184735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.185029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.185312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.191847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.192419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.192613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 03:52:26.192686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.192751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.192813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.192873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.193931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.194925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.195888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 03:52:26.229786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56108843d9c0 pid: 81460 (default app) Aug 31 03:52:26.232928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56108843d9c0 pid: 81459 (default app) Aug 31 03:52:26.233423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc62e8-94b3-4c7c-bc1d-6b786be7792a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:26.233725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc62e8-94b3-4c7c-bc1d-6b786be7792a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:26.233997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc62e8-94b3-4c7c-bc1d-6b786be7792a None None] PATH_INFO: `/v3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:26.238300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Aug 31 03:52:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:32.845049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:32.845628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:32.845628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:32.849961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:32.856129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:32.856710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:32.856710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:32.926618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: INFO dbcounter [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] Registered counter for database keystone Aug 31 03:52:32.947919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG oslo_db.sqlalchemy.engines [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81460) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 03:52:32.957842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writer thread running {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 03:52:33.142588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG passlib.handlers.bcrypt [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81460) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 03:52:33.142882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG passlib.handlers.bcrypt [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81460) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 03:52:33.202740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:33.205507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:33.353275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 03:52:33.882397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:33.961427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: INFO dbcounter [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] Registered counter for database keystone Aug 31 03:52:33.986629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG oslo_db.sqlalchemy.engines [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81459) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 03:52:33.998010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writer thread running {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 03:52:34.117343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013a29-c519-4f16-ad2a-9ba79f66a342 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:34.123120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:34.123728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:34.124006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:34.124271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:34.133578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:34.185888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 03:52:34.186227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8de0-0e20-41f9-86e9-f5c24d4405d0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:34.196598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 31 03:52:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 03:52:34.203878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:34.225061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed230e-eaf3-4f2c-a47b-eb711dfdd01b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:34.231629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:34.232747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:34.233032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:34.233316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] PATH_INFO: `/v3/projects/a624bee162274d80bf52fd9ee90c7807` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:34.235267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] RBAC: Authorizing `identity:get_project(project_id=a624bee162274d80bf52fd9ee90c7807)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:34.288786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING keystone.common.rbac_enforcer.enforcer [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 03:52:34.289144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb78ff-e8ca-4b7f-94ad-c7d7e28d4cb8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:34.293897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Aug 31 03:52:34 2022] GET /identity/v3/projects/a624bee162274d80bf52fd9ee90c7807 => generated 344 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:36.158569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47686df-3ab6-4208-b98d-69d4c531d64b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:36.159492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47686df-3ab6-4208-b98d-69d4c531d64b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:36.159492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47686df-3ab6-4208-b98d-69d4c531d64b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:36.166368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:36.172042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:36.172578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:36.172659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:36.193587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:36.195130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:36.204547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 03:52:36.468889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47686df-3ab6-4208-b98d-69d4c531d64b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:36.490630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47686df-3ab6-4208-b98d-69d4c531d64b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:36.498671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:36.499721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:36.500199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:36.500650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:36.509434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:36.511280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cde027-c07b-484c-934a-96cd28c33fe6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:36.520661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 31 03:52:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 03:52:36.529614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:36.540175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e86177-a74a-47fe-8e82-557da1eab437 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:36.546273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:36.546905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:36.547224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:36.547497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:36.549143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:36.551009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5f8a4-7d3d-44ff-ae90-51d189b41551 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:36.552593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Aug 31 03:52:36 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:38.533368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9105d8-83aa-4d2f-b887-4f07bc3656e5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:38.533929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9105d8-83aa-4d2f-b887-4f07bc3656e5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:38.533929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9105d8-83aa-4d2f-b887-4f07bc3656e5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:38.535376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:38.537569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffea6-9e4a-4691-a462-8b3be5f2d8fa None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:38.537983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffea6-9e4a-4691-a462-8b3be5f2d8fa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:38.538397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cffea6-9e4a-4691-a462-8b3be5f2d8fa None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:38.540541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:38.542676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:38.543086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:38.543489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:38.547180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:38.547680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:38.548293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:38.567477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:38.568661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:38.587642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81459) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 03:52:38.588039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81459) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 03:52:38.637478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:38.639097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:38.651748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 7/12] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:38.743410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 6/13] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:38.912181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:38.937048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.010043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.022180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43be7-b682-4296-b7c8-0eac5a71384b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.057221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.057834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.058117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.058388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.111610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb08ec3-4fc2-44bd-88da-500e27a86ed7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.113918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.125246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.126976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.149508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.149986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.150615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.150897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.151201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.162838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:39.171342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:39.172458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:52:39.173338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:39.178536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.227982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 8/15] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:39.233652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.278395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2c5c8-58c9-431d-81bd-be98d7a58669 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.315136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.315726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.316001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.316267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.316936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:39.317233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.318663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.331300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d59524-6e7f-422e-8ec9-f18705d6ec5e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.331520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 8/16] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:39.337062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.354247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.366472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.366882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.367559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.367970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.368110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.378874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.380377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:52:39.398674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:39.399729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:52:39.400552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:52:39.400552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:52:39.400552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:39.400552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:39.405631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.441062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d803f-e1e9-4bb8-ab60-fb6945e5b4b2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.474415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.475004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.475319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.475591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.477118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.478586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.480380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:52:39.485934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.507323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630ec99-9357-4fcd-b6b3-3452d761abac None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.540386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.541083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.541369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.541642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.542290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:39.542593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.544042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.562163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:39.579722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.587273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c1a94-a992-4469-b68f-f1e4597c0651 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.626455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.627135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.627418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.628259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.629542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.631055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4be3f1-0b89-44f1-9193-193a8c73b450 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.632687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:52:39.639204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:39.684477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:39.756756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d669fa-30a9-44db-901c-e10fdc253b3f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:39.793428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:39.794034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:39.794295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:39.794563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:39.795346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.domain.id=default, user.id=2be81f96a297433a9f04b4dfdb3e8453) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:39.795646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:39.797160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c710b-dbcc-4426-bfc5-cfa31578c9c5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:39.804485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 31 03:52:39 2022] GET /identity/v3/role_assignments?user.id=2be81f96a297433a9f04b4dfdb3e8453&scope.domain.id=default&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:41.891851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7172-5711-472d-bc91-3979968cc232 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:41.892871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7172-5711-472d-bc91-3979968cc232 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:41.893362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7172-5711-472d-bc91-3979968cc232 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:41.894895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:41.900157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19e58-24e5-4528-8651-935077aaeeb3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:41.900991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19e58-24e5-4528-8651-935077aaeeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:41.901462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e19e58-24e5-4528-8651-935077aaeeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:41.924667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-63e19e58-24e5-4528-8651-935077aaeeb3 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:41.926205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-63e19e58-24e5-4528-8651-935077aaeeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:42.015535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 12/23] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:42.033458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf158ec1-3c5f-4c5c-b7a6-9e3790f39d15 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.033739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf158ec1-3c5f-4c5c-b7a6-9e3790f39d15 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.034012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf158ec1-3c5f-4c5c-b7a6-9e3790f39d15 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.035634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 12/24] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:52:42.041199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.041789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.042272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.067556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:42.069085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:42.093337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b709df-0c2b-442a-ace8-7d9228d9813e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.093666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b709df-0c2b-442a-ace8-7d9228d9813e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.093885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b709df-0c2b-442a-ace8-7d9228d9813e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.095529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 13/25] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:52:42.096751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:42.101696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.101993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.102250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.149017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:42.150158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:42.190646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 13/26] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:42.268560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:42.426061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:42.543549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f72ac-3b57-432f-922e-bbd6e93c30e0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:42.571922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:42.594378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:42.595200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.595484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.595754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.618765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:42.625833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:42.626715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:42.627494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:42.633296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:52:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:42.645257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:42.679646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:42.715477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:42.725435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8f072-878a-46b1-973d-9bf726cc6382 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:42.762183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:42.762937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.763336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.763775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.764631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:42.766153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:42.767499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e705c0-e328-467a-a956-7b971418c0b0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:42.804986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:42.805603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.805880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.806150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.806820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:42.807151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:42.808544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb7144-09f3-4c4f-adb9-8871d60ffbaf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:42.821197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 15/29] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:42.822517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319acec-0c14-4b3c-9131-aa71e35ece78 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.822805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319acec-0c14-4b3c-9131-aa71e35ece78 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.823115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319acec-0c14-4b3c-9131-aa71e35ece78 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.824597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 16/30] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:42.825325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319acec-0c14-4b3c-9131-aa71e35ece78 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:42.901558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 15/31] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 31 03:52:42 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:52:42.902703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:42.912771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:42.935570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319acec-0c14-4b3c-9131-aa71e35ece78 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:42.972539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:42.973192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:42.973473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:42.973797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:42.985991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:42.987712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6dcfcb6f-f139-4169-9bf0-518175753b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:52:43.003844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:43.005967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 17/32] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.007984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.017314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851098d6-1c16-4049-8a1b-7b57437b3889 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.034516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:43.035884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:43.053621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.054361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.054645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.054919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.056265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.058009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.060155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 16/33] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:42 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:43.060983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.091505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:43.169370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 18/34] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:43.170328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.176496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438b28-8864-40cd-9be2-8526db791c81 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.211117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.211758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.212104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.212443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.213165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:43.213464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.214855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.239212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 17/35] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:43.239915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:43.245316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.289676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6828748-4e08-4eb1-ac06-e23504448d37 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.335060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:43.353618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.354254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.354532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.354796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.355966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.359925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.362543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 19/36] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:43.368770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.369879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00148a0-14d1-4d73-b393-c38318cd1471 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.422053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef362b-4509-4706-a314-623a510057aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.423327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef362b-4509-4706-a314-623a510057aa None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.423836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef362b-4509-4706-a314-623a510057aa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.424305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef362b-4509-4706-a314-623a510057aa None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.425185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef362b-4509-4706-a314-623a510057aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.427364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef362b-4509-4706-a314-623a510057aa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.475201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 18/37] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:52:43 2022] POST /identity/v3/projects => generated 301 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:52:43.477391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abe1d7-e414-4c58-95d4-b2642de7922c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.477922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abe1d7-e414-4c58-95d4-b2642de7922c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.478449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abe1d7-e414-4c58-95d4-b2642de7922c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.480084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 19/38] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:43.494528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f416-dd73-4ea4-8295-92dd761c6f07 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.495225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f416-dd73-4ea4-8295-92dd761c6f07 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.495776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5f416-dd73-4ea4-8295-92dd761c6f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.531708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4844b-e05b-4b53-801d-302a2804da0e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.531895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-afa5f416-dd73-4ea4-8295-92dd761c6f07 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:43.533431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afa5f416-dd73-4ea4-8295-92dd761c6f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:43.576134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:43.581722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.583454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.583734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.584008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] PATH_INFO: `/v3/domains/default/users/2be81f96a297433a9f04b4dfdb3e8453/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.586125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2be81f96a297433a9f04b4dfdb3e8453, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.587692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3c569-1c20-4a2d-a686-028695406055 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.611730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 20/39] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:43.616600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49094ab-fa7f-4fdf-8989-37df1a2732a5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.617155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49094ab-fa7f-4fdf-8989-37df1a2732a5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.617650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49094ab-fa7f-4fdf-8989-37df1a2732a5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.619525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 21/40] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:43.620507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49094ab-fa7f-4fdf-8989-37df1a2732a5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.693896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:43.716573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 20/41] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Aug 31 03:52:43 2022] PUT /identity/v3/domains/default/users/2be81f96a297433a9f04b4dfdb3e8453/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:52:43.718384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.718654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.718929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.747154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49094ab-fa7f-4fdf-8989-37df1a2732a5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:43.791734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:43.792848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:43.793410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:43.793889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:43.794811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:43.796532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:43.820563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 22/42] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 31 03:52:43 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:52:43.852605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:43.854332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:43.918112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:43.940288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:44.032270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f53560-849f-40ed-8f5c-bf836fea7504 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:44.058251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:44.071070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:44.071714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:44.071996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:44.072268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:44.073399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:44.074862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:44.076712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 23/44] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:52:44.082289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:44.184968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf25a48-6d59-4219-ae1b-d96bef52659c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:44.206038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:44.213923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:44.223690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:44.224307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:44.224588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:44.224860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:44.225530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:44.227240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41517c0c-1379-4e37-8af6-022a43788ef8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:44.263051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:52:44 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:52:44.335619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c08a22-6f9f-490e-884a-d8b9238afe14 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:44.372447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:44.373075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:44.373353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:44.373627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:44.374301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:44.375900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b169-1211-4941-a776-78290910a0b3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:44.393690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 31 03:52:44 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:52:45.684065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1a228-3b31-4b7c-b4bf-c33c2fc9c5a0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:45.684589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1a228-3b31-4b7c-b4bf-c33c2fc9c5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:45.684589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1a228-3b31-4b7c-b4bf-c33c2fc9c5a0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:45.685932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:45.691873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f332dd-ef1e-43b4-b8b5-1a4c5ec8ee40 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:45.692279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f332dd-ef1e-43b4-b8b5-1a4c5ec8ee40 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:45.692429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f332dd-ef1e-43b4-b8b5-1a4c5ec8ee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:45.712725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-86f332dd-ef1e-43b4-b8b5-1a4c5ec8ee40 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:45.713937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-86f332dd-ef1e-43b4-b8b5-1a4c5ec8ee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:45.764454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:45.787057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:46.054907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1a228-3b31-4b7c-b4bf-c33c2fc9c5a0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.143189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a348a-262e-4818-8419-29e04ed26a25 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.143452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a348a-262e-4818-8419-29e04ed26a25 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.143772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a348a-262e-4818-8419-29e04ed26a25 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.145235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 26/49] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:52:46.150036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.150321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.150570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.162124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1a228-3b31-4b7c-b4bf-c33c2fc9c5a0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.169277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:46.170488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:46.197452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.198078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.198360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.198636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.208851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:46.213685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:46.214513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:46.215316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 24/50] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:46.221278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.239969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 27/51] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:46.252282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:46.325090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0d55-fc54-46ce-b1e7-b22c59e775e7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.359166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.359821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.360098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.360371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.361034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:46.361358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.362749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.371650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:52:46.377411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.474874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680c79-e98a-4122-828d-225e464d3760 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.485938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:46.502362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.516322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.516958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.517646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.517920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.519478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.521252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.523054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:46.528266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.600791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:46.608134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceda6c0-4f9b-4101-bae5-b02b18d570af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.629943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136df51f-7400-4689-9849-bb0642786f60 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.643936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.644568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.644845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.645113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.646300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.647789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.649360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:46.657301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.671639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.672429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.672755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.673072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.674305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.675807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.682251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:46.683372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:46.732136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:46.764310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956c78-9016-4410-98de-8609746ffdc6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.785712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc03c3-cad4-4617-a4ad-43cd4e536905 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:46.796964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.797612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.797894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.798162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.798811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.800299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3f1c3-34b4-4ec2-b456-6b46bd460233 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.818898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:46.819532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:46.819820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:46.820090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:46.820815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.domain.id=default, user.id=2be81f96a297433a9f04b4dfdb3e8453) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:46.821205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:46.822705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217a5d7-4f55-42ab-9641-5de3eef1cfb6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:46.827014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 27/56] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 31 03:52:46 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:52:46.831946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 30/57] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 31 03:52:46 2022] GET /identity/v3/role_assignments?user.id=2be81f96a297433a9f04b4dfdb3e8453&scope.domain.id=default&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:48.757801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab82a4-f0c9-4bdb-af95-c18fcf986c34 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:48.758670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab82a4-f0c9-4bdb-af95-c18fcf986c34 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:48.758670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab82a4-f0c9-4bdb-af95-c18fcf986c34 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:48.760902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:48.768445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:48.768838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:48.769060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:48.789905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:48.791294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:48.869522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:49.185482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab82a4-f0c9-4bdb-af95-c18fcf986c34 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:49.205488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:49.315753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab82a4-f0c9-4bdb-af95-c18fcf986c34 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:49.357021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:49.357629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:49.357929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:49.358196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:49.359372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:49.360951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66078561-af56-4f60-93b0-ad6a428f10f1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:49.362685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:49.370771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:49.551008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:49.630787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c69f4-22ab-4650-849a-0790dd9421b7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:49.667678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:49.668293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:49.668576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:49.668848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:49.669528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:49.671005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15b9b-effc-4100-8f6c-006e41d95b40 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:49.703721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:52:49 2022] POST /identity/v3/users => generated 275 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:52:52.699144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415ebb-fc77-4ba5-9128-e3a813b70b20 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.699662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415ebb-fc77-4ba5-9128-e3a813b70b20 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.699662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415ebb-fc77-4ba5-9128-e3a813b70b20 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.701053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:52.707825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48b1e-0f93-4cd8-877c-2684e293b918 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.708712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48b1e-0f93-4cd8-877c-2684e293b918 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.709174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48b1e-0f93-4cd8-877c-2684e293b918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.737102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-76d48b1e-0f93-4cd8-877c-2684e293b918 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:52.738614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-76d48b1e-0f93-4cd8-877c-2684e293b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:52.764414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ddd98-ca60-496e-82aa-8292afc6bcae None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.765014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ddd98-ca60-496e-82aa-8292afc6bcae None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.765501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ddd98-ca60-496e-82aa-8292afc6bcae None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.767612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 31/63] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:52.778355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.778919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.779405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.811740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:52.813034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:52.862672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 33/64] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:52.883897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1f921-de5c-4adc-8675-5e1daa712be8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.884218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1f921-de5c-4adc-8675-5e1daa712be8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.884457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1f921-de5c-4adc-8675-5e1daa712be8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.886000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 34/65] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:52:52.895187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:52.895469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:52.895747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:52.922757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:52.924243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:52.925325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 32/66] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:52.946063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.016029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 35/67] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:52:53.227578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.242891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.316183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.373915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e123f3f-df3a-45b6-b6dc-1055b833c94c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.409702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.410544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.410956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.411375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.422539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bcd4dbe6-dc15-456c-94b0-e1a0d9358b4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:53.427822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:53.428857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:53.429594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:53.430153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 33/68] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:53.431853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24dff0-ede0-4d1b-aa55-eeb872936e82 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.432143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24dff0-ede0-4d1b-aa55-eeb872936e82 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.432423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24dff0-ede0-4d1b-aa55-eeb872936e82 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.433851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 34/69] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:53.434945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f29f-3751-4e03-bcef-2fe35321185b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.435277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f29f-3751-4e03-bcef-2fe35321185b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.435558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f29f-3751-4e03-bcef-2fe35321185b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.436878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 35/70] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:53.437442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485f29f-3751-4e03-bcef-2fe35321185b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.441638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25ab5e-7ef1-43af-93ae-cba8ef887633 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.475880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.476509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.476788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.477060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.487169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:53.492052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:53.493007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:52:53.493746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 36/71] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:52:53.494345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.512059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.515760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.547763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485f29f-3751-4e03-bcef-2fe35321185b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.582710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.583581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.583858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.584130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.593662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9b83-c2aa-4244-a74e-8d5a4b9395ec None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.594431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a817cd05-5b75-4373-9609-fca3e0812d6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:53.599462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:53.600343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:53.601122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 36/72] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:53.601761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.601940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.602239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.620501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:53.622048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:53.633247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.633858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.634157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.634429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.635117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:53.635422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:53.636822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7197a7-e199-4887-8c7d-b4fe8e91aac8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:53.645505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 37/73] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:53.646669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.646955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.647264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.664473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:53.665676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:53.693807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 37/74] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:53.694659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.747053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 38/75] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:53.747053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.762839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.772645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:53.827956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cad70-fd8f-451c-bf18-b9d6cd69d2f5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.859402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee699b5-4cbf-4805-b930-9083eeaa83da None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:53.874757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.875474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.875754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.876019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.876681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:53.876974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:53.878382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:53.887331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 38/76] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:53.892666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.906474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:53.907682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:53.908158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:53.909069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:53.910200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:53.910694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:53.912918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:53.925018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 39/77] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:53.926486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:53.993813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f66-d2a2-4bea-b7be-dafa0df58f07 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.006429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.020613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.032425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.033063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.033347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.033622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.038581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeeaef-65e2-41a2-a89d-4b1cb278b63c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.046668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.048292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.049954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 39/78] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:54.054778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.072587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.073240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.073510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.073778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.074924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.076363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.078126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 40/79] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:53 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:54.078726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.157362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d38f8-a418-4322-af6f-847a357ec06c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.182132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a7751-2418-4199-89d5-069021113784 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.193525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.194154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.194434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.194714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.195871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.197369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.198963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 40/80] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:54.199737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.222926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.223580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.223881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.224146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.225235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.226697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.228290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 41/81] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:54.233484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.242298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.251242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.301878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b504c-8597-4896-97f5-abf8530259f5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.339651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.340508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.340789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.341060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.350189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d3660-cd3a-48ee-b97c-2c538dc13a58 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.351195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:54.355938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:54.356748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:52:54.357514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 41/82] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:54.360006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.389699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee837d-388f-4927-9074-00072567b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.391238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee837d-388f-4927-9074-00072567b722 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.391737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee837d-388f-4927-9074-00072567b722 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.392210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee837d-388f-4927-9074-00072567b722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.465118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee837d-388f-4927-9074-00072567b722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.466832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee837d-388f-4927-9074-00072567b722 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.473971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278e51-165d-42fd-bf54-13fdb42df6bf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.475877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.485731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.510198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.510808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.511112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.511390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.571900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 42/83] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:54.572760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee837d-388f-4927-9074-00072567b722 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.594909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.596379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.673424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee837d-388f-4927-9074-00072567b722 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.680691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.695319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 42/84] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:54.696133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.709012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.709680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.710139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.710453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.710694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.776718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.778191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.796711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a430-3f7c-4806-b1db-297ff5656a64 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:54.829879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:54.830452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:54.830727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:54.830995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:54.831667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:54.833108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:54.875577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 43/85] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:52:54 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:52:54.875577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.898992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 43/86] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:54.899917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:54.909635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.943143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:54.985004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c45-ad2a-473d-8572-6e193d7bd084 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.015203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fb66-a199-4544-a5da-7f9fb101c2db None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.026336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.026532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.026788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.027092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.027745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:55.028042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.029457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.040659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 44/87] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.041486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.054838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.055612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.055896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.056171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.057386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.058965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.060910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 44/88] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:52:54 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.061429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.152736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:55.173660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f3e7d-a02b-44bb-842a-41bb034fef20 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.180763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413131a-c22e-4c0d-89f9-e70ad1f22557 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.204222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:55.216312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.216951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.217331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.217615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.219005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.220828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.221055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.221472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.221755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.222031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.222821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 45/89] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.223525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.231081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.231242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.231242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 45/90] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.231242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.352973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e4c35-7dce-4239-929e-778ee17a9721 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.368926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d71e1c-8833-48ba-9fcd-1df54057d3d0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.390835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.391914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.392459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.392832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.394419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.396012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.398504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 46/91] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.399678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.413027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.413859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.414720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.415190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.416497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:55.416943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.419140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.426306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 46/92] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.427330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.444915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:55.519678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:55.596267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bd723-db63-4570-93dc-8abd5388f9e5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.616612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b971f2-be92-4897-97ad-9b6a32004d15 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.655755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.656775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.657118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.657478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.658267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:55.658677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.660259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.666611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 47/93] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=ee67389746c245038655ca9ddc941126 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.667406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:55.679598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.680237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.680517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.680904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.681663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=2be81f96a297433a9f04b4dfdb3e8453) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:55.682024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.684257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:55.696716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/role_assignments?user.id=2be81f96a297433a9f04b4dfdb3e8453&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:55.784235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:55.827747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522716-70c2-474d-bdcc-9ecc019ce4d3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:55.872708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:55.873339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:55.873622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:55.873897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:55.970743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:55.972256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:56.061652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:56.103919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 48/95] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:56.109649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:56.240251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94d4e-6144-46d4-b35a-9160c4e9c88f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:56.269627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:56.276807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:56.277460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:56.277746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:56.278033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] PATH_INFO: `/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:56.279196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=28b1ceef9f694ba3be9bdb17bcc4adc8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:56.281060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687f90-4822-4318-a7a8-5121a7a73aa6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:56.282929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:52:56 2022] GET /identity/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:56.292156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:56.420987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3adf-e3d7-435c-a0e7-2bdbfd4e6d67 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:56.456058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:56.456665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:56.456944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:56.457216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:56.457948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=28b1ceef9f694ba3be9bdb17bcc4adc8, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:56.458253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:56.459727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0b6b-2831-4e29-9d5d-ad6c8208cd4d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:56.465216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:52:56 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=28b1ceef9f694ba3be9bdb17bcc4adc8&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:57.485059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4932fb-5d7a-4163-bf37-0fd31ebc2424 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:57.485059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4932fb-5d7a-4163-bf37-0fd31ebc2424 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:57.485059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4932fb-5d7a-4163-bf37-0fd31ebc2424 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:57.485059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:57.491086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:57.491869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:57.492349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:57.511257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:57.512751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:57.540269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:57.558108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143a726-0b11-4d48-a3d6-02286e3cacf2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:57.558383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143a726-0b11-4d48-a3d6-02286e3cacf2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:57.558660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143a726-0b11-4d48-a3d6-02286e3cacf2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:57.560357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 50/99] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:52:57.568305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:57.568805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:57.569244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:57.597973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 50/100] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:57.605978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:57.607422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:57.721795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 51/101] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:57.990062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:58.113651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad7949-659e-40fe-b98b-e9a5535fe752 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:58.155718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:58.156794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.157281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.157741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.159313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:58.160964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:58.162851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 51/102] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:52:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:58.168728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:58.247882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:58.301393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:58.344492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311419a0-876f-49f6-b8cd-c7171350a171 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:58.403570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:58.419059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:58.419738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.420019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.420494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.421177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:58.422976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d456c19-96af-4c35-8076-9d369f5dbe20 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:58.436988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d9531-ae5d-4638-9c20-43f0baf96fda None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:58.489130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 52/103] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:52:58 2022] POST /identity/v3/projects => generated 302 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:52:58.490423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931c35-a0b1-460f-b25c-9ab2fd6f6c06 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.491602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931c35-a0b1-460f-b25c-9ab2fd6f6c06 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.491886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931c35-a0b1-460f-b25c-9ab2fd6f6c06 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.496551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 53/104] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:58.502232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:58.502850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.503128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.503423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.505435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2755a-1a51-4e23-a9a4-0051bc1cc8fc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.505723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2755a-1a51-4e23-a9a4-0051bc1cc8fc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.506000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2755a-1a51-4e23-a9a4-0051bc1cc8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.529508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:58.542005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:58.542906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:52:58.544280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 52/105] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:58.545812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-dce2755a-1a51-4e23-a9a4-0051bc1cc8fc None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:58.547322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dce2755a-1a51-4e23-a9a4-0051bc1cc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:58.551609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:58.644000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 54/106] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:58.645659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23f3b5-b17f-424c-b1e9-35d6581c1e1e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.646123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23f3b5-b17f-424c-b1e9-35d6581c1e1e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.646551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23f3b5-b17f-424c-b1e9-35d6581c1e1e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.652820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 55/107] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:58.658416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.658872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.659338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.689766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:58.691302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:58.715605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:58.753338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d48cf-e741-447c-b84f-d1f7bdce2f7b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:58.778592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 56/108] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:58.828323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:58.829103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:58.829386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:58.829665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:58.830345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:58.830654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:58.832081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:58.853157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 53/109] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:52:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:58.859873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:58.925192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:58.997923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e03ea2-79a8-4bf5-8786-1bd8a728238e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.008894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.034793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.035819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.036373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.036865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] PATH_INFO: `/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.055718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] RBAC: Authorizing `identity:get_user(user_id=1a58a6afdc704fe0b9dd7907ab180f1e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.057530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.059841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 57/110] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:58 2022] GET /identity/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:59.066067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.190988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db37bb-b634-4e81-ab8f-6e076f7ee39c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.206711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.212656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b0d3-8032-4bb0-94f7-fe81cc1854ce None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.236982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.237950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.238239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.238777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.248919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.249526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.249804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.250105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.251457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:59.257873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:59.258733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:52:59.259561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:52:59.263161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 54/111] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:59.264009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.286134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.334061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.335599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06204fdf-747c-4943-8083-c4b3691a87f1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.377633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e39352-804c-4448-834f-1f45bfb3d9d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.414021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.414655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.415460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.415733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.428849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:52:59.437448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bcc83714-ab40-47b7-b859-3e3a4084fc5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:52:59.438297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:52:59.439134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:52:59.440178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 55/112] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:52:59.443071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447b33-0dfc-4b5c-8d53-ec75fd4d2807 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.443071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447b33-0dfc-4b5c-8d53-ec75fd4d2807 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.443071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447b33-0dfc-4b5c-8d53-ec75fd4d2807 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.444569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 56/113] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:59.445282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10447b33-0dfc-4b5c-8d53-ec75fd4d2807 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.446907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 58/114] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:52:59.448190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed96b7-a359-4f04-88d5-1c6cabf78bd9 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.448469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed96b7-a359-4f04-88d5-1c6cabf78bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.448740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed96b7-a359-4f04-88d5-1c6cabf78bd9 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.450161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 59/115] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:52:59.450673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed96b7-a359-4f04-88d5-1c6cabf78bd9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.540487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.556998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed96b7-a359-4f04-88d5-1c6cabf78bd9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.569023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10447b33-0dfc-4b5c-8d53-ec75fd4d2807 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.595485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.596096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.596373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.596646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.597322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:59.597597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.598994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.605200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.606013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.606464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.606908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.607401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 60/116] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:59.607941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:52:59.608175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.608590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.610637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1e27c-3486-40e5-9d15-2a27fbf95ff2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.619597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 57/117] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:59.620767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b85bf9-b519-4bf6-9df8-720b2de56ac5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.621055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b85bf9-b519-4bf6-9df8-720b2de56ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.621637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b85bf9-b519-4bf6-9df8-720b2de56ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.640257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a1b85bf9-b519-4bf6-9df8-720b2de56ac5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:59.642054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a1b85bf9-b519-4bf6-9df8-720b2de56ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:59.677690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.717759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 58/118] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:59.723062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.723062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.723062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.730773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d20530-43c8-44ee-b881-bda250152b4a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.739659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:52:59.740499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:52:59.777086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.777954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.778366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.779045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.781262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.783019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.785154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 61/119] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:59.786701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.807305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:52:59.817944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 59/120] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:52:59.818946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:52:59.933790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93892d2-c11b-4003-921b-aff8b676cb1e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.972453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:52:59.973146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:52:59.973468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:52:59.973751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:52:59.976809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:52:59.977078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb62f-cd39-4cd1-9da7-33f883a4d6d2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:52:59.978263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:52:59.980508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 60/121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:52:59.981279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.009284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.018645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.019323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.019612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.019894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.021056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.022727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.024385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 62/122] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.029672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.050100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.091370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712c66b-83e3-49ad-a582-f339125558b8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.126935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.127862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.128272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.128697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.129555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=1a58a6afdc704fe0b9dd7907ab180f1e) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:00.130044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.130284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e42c7c-7179-4e61-84fd-e38d08f66cad None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.131879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.137843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 61/123] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:52:59 2022] GET /identity/v3/role_assignments?user.id=1a58a6afdc704fe0b9dd7907ab180f1e&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=a856bfff6600429a90907865739b465b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.138997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.186896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.187937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.187937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.188206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.201563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:00.207344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:00.208363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:00.209104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 63/124] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:00.210276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.277356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.288512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87ee44-3df7-41d6-91e5-936e10ee7590 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.324438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.330072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090412f2-dc53-4421-a685-9e12744f7df0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.334476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.335141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.335422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.335692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.336820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.338271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.340125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 62/125] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.340786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.369701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.370783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.371663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.372097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.373396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.375481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.377664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.378520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.464168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235245f-d313-4d85-9722-f139aadf3c0e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.488161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dd906-e7bf-4c2d-81e0-be7f80931043 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.511785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.512976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.513471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.514038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.515527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.527293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.528058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.528429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.528791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.529550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:00.529948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.530611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:00.531659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:00.537758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:00.538822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:00.539798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:00.542058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:00.547738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.547738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.548083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.606789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.655272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb89e3-3859-4ac7-83c1-76d495d1e579 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.674045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421269-57ae-4fe0-863e-be09c2cab1fa None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.691624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.692272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.692550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.692822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/users/ca8683acefdd46fa801a7427335c1529/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.694742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, user_id=ca8683acefdd46fa801a7427335c1529, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.696252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.715101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 64/129] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:00 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/users/ca8683acefdd46fa801a7427335c1529/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:00.716127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.718364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.719350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.719862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.720420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/users/ca8683acefdd46fa801a7427335c1529/roles/ee67389746c245038655ca9ddc941126` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.722989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, user_id=ca8683acefdd46fa801a7427335c1529, role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.724801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.745372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 66/130] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:00 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/users/ca8683acefdd46fa801a7427335c1529/roles/ee67389746c245038655ca9ddc941126 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:00.747049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.833096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.871663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a412e2-91b6-442e-b88a-ad4f5ca479b4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.890849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:00.903254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f7a72-3922-4c94-b162-7a2a8b51b56a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:00.912854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.913823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.914312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.914771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.915667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:00.916159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.917773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7075be-3a00-43ec-a53a-fa14d26867c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.927922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.934078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b1f3-ca06-4837-a685-a8f04c95380d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.934815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b1f3-ca06-4837-a685-a8f04c95380d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.935330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192b1f3-ca06-4837-a685-a8f04c95380d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.937102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 66/132] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:00.938098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192b1f3-ca06-4837-a685-a8f04c95380d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:00.946409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:00.947109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.947389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.947662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.948845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:00.950803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fb13e-55fd-4f8f-8e15-7cd3acd801bb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:00.952250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 67/133] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:00.953705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:00.954414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:00.954414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:00.973687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:00.974993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:01.048716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 68/134] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:01.049792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.057667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192b1f3-ca06-4837-a685-a8f04c95380d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.092887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.093995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.094520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.094983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.096454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.098129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.099517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:01.100210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:00 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:01.106250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.155688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:01.163224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600d64e-902d-4273-bd91-1a2fb77c593e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.199150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.199849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.200132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.200406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] PATH_INFO: `/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.201656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] RBAC: Authorizing `identity:get_project(project_id=28b1ceef9f694ba3be9bdb17bcc4adc8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.203294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.205440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 69/136] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:01 2022] GET /identity/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:01.210649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.215325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097473a-c268-4123-a5cf-e54b03f705fe None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.250974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.251992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.252457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.253067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.254390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.256073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.257995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 68/137] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:01 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:01.263694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.330480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a255c-7882-4eac-8039-b024c6dee00e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.341805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:01.368905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.369569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.369843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.370110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] PATH_INFO: `/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8/users/ca8683acefdd46fa801a7427335c1529/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.372415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] RBAC: Authorizing `identity:create_grant(project_id=28b1ceef9f694ba3be9bdb17bcc4adc8, user_id=ca8683acefdd46fa801a7427335c1529, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.373879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.375562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4401a-5090-4227-b41f-4e982c4561bc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.390375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 70/138] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:01 2022] PUT /identity/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8/users/ca8683acefdd46fa801a7427335c1529/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:01.400056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:01.426972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.428067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.428575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.429124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/users/2be81f96a297433a9f04b4dfdb3e8453/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.431694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, user_id=2be81f96a297433a9f04b4dfdb3e8453, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.433492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.450036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 69/139] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:01 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/users/2be81f96a297433a9f04b4dfdb3e8453/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:53:01.584592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.703893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127d9b9-21d6-406f-a81b-ea331268597e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.744407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.745317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.745731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.746573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.748852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.750524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3897fc-a00b-4c37-b7fc-66a4f9516cec None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.752395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 71/140] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:01.758192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:01.886620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c83b50-5e34-45df-913c-a9e1adc6f85a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:01.926514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:01.927166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:01.927454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:01.927719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:01.928380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:01.929891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7276-1168-430e-a260-03047812f08c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:01.960633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 70/141] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:01 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:02.745316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c5da-79d6-41ef-ae97-7d64b3e3a785 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:02.746635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c5da-79d6-41ef-ae97-7d64b3e3a785 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:02.747132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c5da-79d6-41ef-ae97-7d64b3e3a785 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:02.749128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 72/142] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:02.754159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:02.754447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:02.754736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:02.783137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:02.784413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:02.814214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:02.873275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 71/143] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:03.284523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c5da-79d6-41ef-ae97-7d64b3e3a785 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:03.339236 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:03.411776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c5da-79d6-41ef-ae97-7d64b3e3a785 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:03.460021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:03.461024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:03.461497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:03.461965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:03.478294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:03.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:03.484333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:03.485230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:03.487541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 73/144] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:03.493317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:03.611484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630b75-50ab-44a7-8349-af85f6dc716a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:03.651757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:03.652740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:03.653218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:03.653687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:03.654575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:03.655119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:03.656798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8ef7a-7703-4046-b352-a7dacf5b1292 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:03.668185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 72/145] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:03.674144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:03.784913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573522b-aec8-41c0-9af8-5ea289026dff None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:03.785918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573522b-aec8-41c0-9af8-5ea289026dff None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:03.786383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573522b-aec8-41c0-9af8-5ea289026dff None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:03.788067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 73/146] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:03.794147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665303b-e886-42d5-b114-fc941f8ce912 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:03.794693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665303b-e886-42d5-b114-fc941f8ce912 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:03.795214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665303b-e886-42d5-b114-fc941f8ce912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:03.804870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:03.810527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049a45c-ed07-4a4e-9850-86d93efdc3ce None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:03.820649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b665303b-e886-42d5-b114-fc941f8ce912 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:03.821864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b665303b-e886-42d5-b114-fc941f8ce912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:03.855515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:03.858543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:03.859231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:03.859572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:03.859845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] PATH_INFO: `/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:03.862890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] RBAC: Authorizing `identity:get_user(user_id=1a58a6afdc704fe0b9dd7907ab180f1e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:03.864337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:03.865991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:03 2022] GET /identity/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:03.872023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:03.966460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 74/148] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:04.045545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6be2e8-40de-40c2-af56-52f81898cd1a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:04.079358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4485-d1da-4762-9f49-4d0bf014452c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.079660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4485-d1da-4762-9f49-4d0bf014452c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.079896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4485-d1da-4762-9f49-4d0bf014452c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.081268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 75/149] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:04.090133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:04.090770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.091062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.091591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.092730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:04.092829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdb38f-ef18-4392-ba13-d6023894a23f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.093053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdb38f-ef18-4392-ba13-d6023894a23f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.093328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdb38f-ef18-4392-ba13-d6023894a23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.094348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:04.096080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 75/150] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:03 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:04.101649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:04.124893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7abdb38f-ef18-4392-ba13-d6023894a23f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:04.126080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7abdb38f-ef18-4392-ba13-d6023894a23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:04.151640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:04.224443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d010a5a-03f4-4e43-b8e4-2e210b854579 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:04.275763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:04.276451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.276730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.276999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/users/1a58a6afdc704fe0b9dd7907ab180f1e/roles/a856bfff6600429a90907865739b465b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.278987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, user_id=1a58a6afdc704fe0b9dd7907ab180f1e, role_id=a856bfff6600429a90907865739b465b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:04.285364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71c62-07b0-4386-8673-06cce873257d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:04.296381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 76/151] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:04.308030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 76/152] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:04 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/users/1a58a6afdc704fe0b9dd7907ab180f1e/roles/a856bfff6600429a90907865739b465b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:04.512904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f87f1-63f9-4d18-b9d2-50d6dae97470 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.513282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f87f1-63f9-4d18-b9d2-50d6dae97470 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.513450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f87f1-63f9-4d18-b9d2-50d6dae97470 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.514822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 77/153] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:04.533004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.533303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.533541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.545000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b082f-2e49-418c-b504-3a17ded8c0a2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.545324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b082f-2e49-418c-b504-3a17ded8c0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.545558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b082f-2e49-418c-b504-3a17ded8c0a2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.547322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 78/154] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:04.560400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:04.561638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:04.565272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.565555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.565827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.594777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:04.596025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:04.658954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 77/155] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:04.660074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:04.693042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:04.762846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 79/156] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:04.785734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a34a62-910b-4d29-90c8-7918c0470a25 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:04.826399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:04.827072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:04.827328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:04.827597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:04.845341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:04.850176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:04.851001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:04.851789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 78/157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:04.858902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:04.890150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:04.933992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.028162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8347-1106-4ad2-a61f-ffc27a5ccd56 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.039058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.057449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a02a7-b0a5-4597-8ad7-8a87603de5e9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.073363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.073984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.074271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.074548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.083449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:05.083774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:05.085184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:05.093812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 80/158] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:05.098363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.099071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.099351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.099614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.099898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.110954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:05.119606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:05.120659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:05.121745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 79/159] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:05.126887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.202224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.234311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317a16-8b85-4964-9bb9-19461808a3a6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.268759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.269403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.269682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.269959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.271142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:05.272596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:05.274392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 81/160] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:05.274982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.305115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c551d0d-0c68-4b14-ba69-43b46da7e487 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.371677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.372374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.372652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.372915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.373583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:05.373877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:05.375412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:05.392400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 80/161] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:05.393394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.404996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce2950-1ac0-4269-85cb-6d4ad2ff4bb5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.406048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.444968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.445590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.445871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.446147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.465370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:05.474353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:05.475357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 82/162] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:05.476141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.490299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.552700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1209d-a8ec-4326-83a4-56007b78e0aa None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.615881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.616446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.616723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.616993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.633523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28905f9b-98b8-4220-996e-36efc09614d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.687481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.688723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.689360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.689829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.692923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:05.694668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:05.700868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 83/163] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:05.701998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.772107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:05.773616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:05.787255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.840953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:05.905995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f091b3-6de8-4fe0-af34-2579038f2631 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:05.963619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 81/164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:05.964713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:05.974150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:05.974789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:05.975084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:05.975365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:05.986677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:05.997035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:05.997957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:05.998724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 84/165] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:06.005301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.165894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7440ad7-ca07-4f75-862c-29ce9c07268e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.176682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.223853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270bd47-5d07-4d75-8178-10f506d23146 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.241949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:06.242568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.242849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.243148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.243820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:06.244120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:06.247795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:06.273297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 82/166] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:06.274171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.277911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.284860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211171a2-160a-431d-906d-e352c2221608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:06.285480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211171a2-160a-431d-906d-e352c2221608 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.285823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211171a2-160a-431d-906d-e352c2221608 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.286101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211171a2-160a-431d-906d-e352c2221608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.418945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211171a2-160a-431d-906d-e352c2221608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:06.420452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211171a2-160a-431d-906d-e352c2221608 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:06.495081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce8be1-9f0b-4a68-a661-fe914cfd4164 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.529131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 85/167] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:06.530463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211171a2-160a-431d-906d-e352c2221608 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.569256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.585291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:06.585973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.586193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.586464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.591695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:06.593164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff3ac6-4ac0-4155-a39a-991e43ce14af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:06.594949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 83/168] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:06 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:06.596332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb157a-2537-4264-a69f-103fedd384ec None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.596443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb157a-2537-4264-a69f-103fedd384ec None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.596626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb157a-2537-4264-a69f-103fedd384ec None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.598089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 84/169] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:06.598782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb157a-2537-4264-a69f-103fedd384ec None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.651439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211171a2-160a-431d-906d-e352c2221608 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.663852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.707713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:06.708724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.709202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.709671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.715058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:06.715659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:06.717274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:06.734819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 86/170] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:06.736060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.737960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb157a-2537-4264-a69f-103fedd384ec None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.779060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:06.779060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.779060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.779060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] PATH_INFO: `/v3/users/2be81f96a297433a9f04b4dfdb3e8453` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.779477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] RBAC: Authorizing `identity:get_user(user_id=2be81f96a297433a9f04b4dfdb3e8453)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:06.779477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95531bf3-17ad-4b25-a816-a1ef8ecbc472 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:06.779936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 85/171] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:06 2022] GET /identity/v3/users/2be81f96a297433a9f04b4dfdb3e8453 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:06.781047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:06.781814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:06.781814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:06.801371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:06.802739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:06.874708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 86/172] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:06.875718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:06.930908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.947126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bd8a-930a-445a-b8c1-b12c4be717c7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:06.975593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:06.985587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c32b09-ceb8-4780-a6bc-042b23856a1d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.007020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.007686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.007961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.008231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.010715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.012212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.015181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 87/173] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:06 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:07.015886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.053217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.053825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.054103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.054377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.055137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:07.055442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.056912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.068119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 87/174] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:06 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:07.069404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.222537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.226944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1196c-b304-4a58-b41e-0219c05ade07 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.246096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf614e1-e86a-4cb5-b48c-a28c43e534e2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.266904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.267564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.267883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.268108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.293731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.294362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.294635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.294901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] PATH_INFO: `/v3/users/ca8683acefdd46fa801a7427335c1529` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.296052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8683acefdd46fa801a7427335c1529)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.297923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0a3ed-c924-4e34-93f0-ec4e8e2342e6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.299577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 88/175] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/users/ca8683acefdd46fa801a7427335c1529 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:07.300812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb9b3-06bd-46ae-b88f-739cae2fda64 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.301094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb9b3-06bd-46ae-b88f-739cae2fda64 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.301364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb9b3-06bd-46ae-b88f-739cae2fda64 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.302688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 89/176] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:07.304815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb9b3-06bd-46ae-b88f-739cae2fda64 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.333581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.349517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.351109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.407349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb9b3-06bd-46ae-b88f-739cae2fda64 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.439971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.440578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.440862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.441135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.441862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:07.442169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.443647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7169fbd-4cd5-4fe0-a44c-4f3f818f4ca5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.444532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.449546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 90/177] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=ee67389746c245038655ca9ddc941126 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:07.455167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 88/178] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:07.455363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5da2f-9490-495e-9928-b6e048c816f8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.455363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5da2f-9490-495e-9928-b6e048c816f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.455512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.455581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5da2f-9490-495e-9928-b6e048c816f8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.455581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 91/179] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:07.455581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52703922-f856-4420-8932-7213fdfd4564 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.455581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52703922-f856-4420-8932-7213fdfd4564 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.455581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52703922-f856-4420-8932-7213fdfd4564 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.456713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 92/180] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:07.457319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52703922-f856-4420-8932-7213fdfd4564 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.566035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.570110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f7660-9074-4f7c-9627-b215de452de2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.577787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52703922-f856-4420-8932-7213fdfd4564 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.609045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.610083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.610364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.611044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.619653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.620480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.620942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.621678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.622823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9bca89d5-ca97-468a-8dc2-4e3b0b9220e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:07.627986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:07.628915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:07.629725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 89/181] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:07.630391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.630804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.630804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.648986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:07.650178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:07.701983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.703668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7985e-abc0-45df-9982-f02ee442aa42 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.706687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.721561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 90/182] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:07.722404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.808142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.820592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 93/183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:07.821747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.822024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.822396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.831161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ac118-117d-4c1e-afa3-da878a3ada24 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:07.842100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:07.843458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:07.864577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:07.865204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.865480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.865746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.866859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:07.868345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad090cd5-435b-4b09-9dd2-a68a51fa9153 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:07.869992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 91/184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:07.871096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397610e8-5835-4208-954b-604041c7a322 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:07.871679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397610e8-5835-4208-954b-604041c7a322 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:07.871944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397610e8-5835-4208-954b-604041c7a322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:07.891070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-397610e8-5835-4208-954b-604041c7a322 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:07.892248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-397610e8-5835-4208-954b-604041c7a322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:07.922513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 94/185] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:07.931290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:07.953453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:07.972333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 92/186] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:07.973375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397610e8-5835-4208-954b-604041c7a322 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.057431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59380f-45fb-498b-a4c1-c6482d24aeaf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.080954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.093886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.094606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.095010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.095440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.096441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:08.096832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.098306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.107372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 95/187] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.108092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.124505 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397610e8-5835-4208-954b-604041c7a322 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.171648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.172968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.173251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.173698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] PATH_INFO: `/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.175328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=28b1ceef9f694ba3be9bdb17bcc4adc8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.177087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.179175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 93/188] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:07 2022] GET /identity/v3/projects/28b1ceef9f694ba3be9bdb17bcc4adc8 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.180407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.216996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e948ce-4a49-4fa5-97cc-623ecc2f29bc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.245257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.252896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.253495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.253765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.254030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.254753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=2be81f96a297433a9f04b4dfdb3e8453) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:08.255079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.256527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.262118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 96/189] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/role_assignments?user.id=2be81f96a297433a9f04b4dfdb3e8453&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.262884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.305819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0adbd-0c6e-4b18-bb8d-0d3d49cd1dd5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.324419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.344904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.345508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.345789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.346065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.357376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:08.363128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:08.364061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:08.364857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 94/190] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:08.365508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.390290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac26e2f-7eda-4ca8-aa93-592ce32e0464 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.431300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.432079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.432371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.432643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.451096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.452537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.454616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 97/191] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.460152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.481437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea9a08-bf54-4e4f-8f3e-eabf709ceeac None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.496535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.518012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.518618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.518896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.519195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.520051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=28b1ceef9f694ba3be9bdb17bcc4adc8, user.id=ca8683acefdd46fa801a7427335c1529) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:08.520356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.521809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.527641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 95/192] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/role_assignments?user.id=ca8683acefdd46fa801a7427335c1529&scope.project.id=28b1ceef9f694ba3be9bdb17bcc4adc8&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.536119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.572134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016a6-d4db-45f1-a5dd-3eb3dd88bd35 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.582658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.613957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.614490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.614767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.615059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.627757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:08.632712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:08.633643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:08.634423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 98/193] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:08.639592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.653978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906db2c6-6ae7-4c08-ab96-8c69e866c064 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.693842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.694474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.694752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.695045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.696158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.697612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.699226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 96/194] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.704596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.761049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.761364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3b5b8-1461-4740-9268-656e6b0b1f34 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.795115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.795719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.795991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.796256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.796942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:08.797238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.798667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.806507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 99/195] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.811748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.813462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d6ce-302c-4ba6-bc25-f6b75b35d979 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.841697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:08.847615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.848224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.848500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.848768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.913467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3730102-caee-4d3c-9b17-1c80b3f2ace5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:08.921305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.923055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.947157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:08.947755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:08.948028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:08.948294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:08.948945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:08.949237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:08.950600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:08.959659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 100/196] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:08.959659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:08.971283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.025308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 97/197] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:09.026193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.050131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.060811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981ce46-3dc0-449e-8c30-3bf8e876f789 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.092899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.093497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.093808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.094078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.094716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.096220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.116662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 101/198] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:08 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:09.117474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.139736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84725aa-2c59-4450-aff0-11b6e9a16cb3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.182429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.183108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.183389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.183653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.184868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.186338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.188141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 98/199] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:09.194192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.210924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.232355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ba2b4-78f9-469c-a65b-abc2b9ee89fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.269841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.270351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.270631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.270916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] PATH_INFO: `/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.272082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=1a58a6afdc704fe0b9dd7907ab180f1e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.273521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.275454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 102/200] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/users/1a58a6afdc704fe0b9dd7907ab180f1e => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:09.276288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.296298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.313488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6576e-1d5c-46cd-85ef-8c535518a020 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.360383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.361023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.361296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.362696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.364923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.366371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.367826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 99/201] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:09.368586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.411639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9533fa-c90b-42d8-ae40-9c2338f5c2a1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.449201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.449769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.450042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.450306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.470267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.495484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b18806-bd8b-4602-9552-5f36cfca8e35 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.533866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.534231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.534470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.534743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.535008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.535694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.551905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.608042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.609841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.643262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 103/202] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:09.644230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.695015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.715824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 100/203] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:09.721292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.751393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be981-7f3d-4050-adcd-eec220e7e82a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.768827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:09.796668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.797341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.797612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.797882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.798882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=d8f8fbe039fe4790a9606b5c699437c4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:09.800086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.801581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.808348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 104/204] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/role_assignments?user.id=d8f8fbe039fe4790a9606b5c699437c4&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:09.808988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:09.854004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cc09b-3ea9-4ea6-a663-89c8f0678762 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:09.908159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:09.909085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:09.909497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:09.909900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:09.912592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:09.914267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:09.917300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 101/205] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:09.923852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:10.016590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:10.020551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472cb6c-0680-4f9d-ae3d-6e8ea625d8f7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:10.067055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:10.067055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:10.067055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:10.067055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:10.068879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:10.072028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:10.075264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 105/206] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:10.087995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:10.109820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f8959-252d-4f67-9d2e-1f597f003778 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:10.148508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:10.149791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:10.150239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:10.150646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:10.151539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=d8f8fbe039fe4790a9606b5c699437c4) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:10.152003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:10.153753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde092df-1d5c-4489-8098-627f8ff6dd2d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:10.160474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 102/207] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:09 2022] GET /identity/v3/role_assignments?user.id=d8f8fbe039fe4790a9606b5c699437c4&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=ee67389746c245038655ca9ddc941126 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:10.177267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:10.235872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ebfc2-3c7d-412a-9796-1c1c58ac96a6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:10.273026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:10.273871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:10.274296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:10.274688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:10.275644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, scope.project.id=f2239116cdae485a95fef3677e13e435, user.id=1a58a6afdc704fe0b9dd7907ab180f1e) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:10.276071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:10.277668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93d78e-49d7-4461-a8ba-c919f59901f9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:10.284714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 106/208] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:10 2022] GET /identity/v3/role_assignments?user.id=1a58a6afdc704fe0b9dd7907ab180f1e&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=a856bfff6600429a90907865739b465b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:11.289384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb6ba-017b-4f73-baef-4813e667183b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.289831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb6ba-017b-4f73-baef-4813e667183b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.290009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb6ba-017b-4f73-baef-4813e667183b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.291690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 103/209] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:11.297166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3da12-265a-4da2-aba4-5703fa6ab2e3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.297592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3da12-265a-4da2-aba4-5703fa6ab2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.297780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3da12-265a-4da2-aba4-5703fa6ab2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.316654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ef3da12-265a-4da2-aba4-5703fa6ab2e3 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:11.317857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ef3da12-265a-4da2-aba4-5703fa6ab2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:11.387424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 107/210] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:11.471527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e099-eeaa-409e-b716-37e4eaf3691c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.472145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e099-eeaa-409e-b716-37e4eaf3691c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.472634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e099-eeaa-409e-b716-37e4eaf3691c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.474216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 104/211] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:11.479748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.480254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.480739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.499461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:11.501001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:11.535154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:11.572551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 108/212] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:11.672576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e099-eeaa-409e-b716-37e4eaf3691c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:11.786366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e099-eeaa-409e-b716-37e4eaf3691c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:11.801230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:11.822795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:11.823451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.823726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.823998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.835469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:11.837015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:11.840398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:11.841280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:11.842466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 105/213] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:11.847783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:11.941145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0dbd5-ddbc-4af0-ba25-bebce9e9bfb8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:11.959405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4a05e-a149-4d76-acac-b99cdf9ab3b2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:11.977432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:11.978064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.978459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.978646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.979742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:11.981195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f321f-82d8-4f8d-9b2b-75e290cbd142 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:11.982994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 109/214] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:11.983990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054de00-dffd-4746-97fa-faf01fe5c14c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:11.984335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054de00-dffd-4746-97fa-faf01fe5c14c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:11.984612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054de00-dffd-4746-97fa-faf01fe5c14c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:11.985962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 110/215] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:11.990562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054de00-dffd-4746-97fa-faf01fe5c14c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.001818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.002426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.002984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.002984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.003632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:12.003930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.005327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdaeeb2-8494-47ca-873b-c02b6177389d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.013957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 106/216] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:12.015621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.016400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.016705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.037520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:12.038701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:12.042452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.078633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.110591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054de00-dffd-4746-97fa-faf01fe5c14c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.126130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 107/217] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:12.127314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.156672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.157329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.157609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.157928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.158585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.160101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f381-5c1b-4b17-9b18-fda1ebd972f3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.185846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 111/218] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:11 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:12.237210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc120a-553c-4162-903a-470bad95c050 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.237561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc120a-553c-4162-903a-470bad95c050 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.237765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc120a-553c-4162-903a-470bad95c050 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.239399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 112/219] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:12.245576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.246501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049b27e-2ab4-459d-967e-4b50c7847fd4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.246714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.247231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.268659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:12.270381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:12.299172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.299828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.300036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.300308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] PATH_INFO: `/v3/users/95594ec7979d467380701c3d20811b30` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.317607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] RBAC: Authorizing `identity:get_user(user_id=95594ec7979d467380701c3d20811b30)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.319206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.321045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 108/220] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/users/95594ec7979d467380701c3d20811b30 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:12.327152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.356050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.361368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 113/221] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:12.395887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.424664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.450515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d191-c2ab-47f9-b5db-6a16d847ee1f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.492415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.493264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.493680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.494144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.512304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca251994-c04c-4248-a0b7-2153440c6877 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.555469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.556049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.556327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.556594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.568961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:12.571844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.573552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:12.574012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:12.574894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:12.575707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 114/222] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:12.586671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.609737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.694894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.696123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 109/223] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 03:53:12.696969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.718152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475048f5-93c2-4559-b734-71a98feb4729 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.759395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.760000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.760277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.760542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.761206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:12.761501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.762890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.772174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 115/224] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:12.773041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.809367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f07ce-7eae-41b8-951b-6c527953efeb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.837025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:12.850303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.851315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.851791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.852251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.864450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:12.870009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:12.870896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:12.872055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:12.874766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 110/225] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:12.876087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.878815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cef4-3f63-418c-a6a5-3c299e854e22 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:12.913767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:12.914403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:12.914684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:12.914956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:12.916117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:12.917572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:12.919561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 116/226] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:12.920335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:12.936568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:13.001659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc11e3-823e-4392-8235-ca27bee9c7c3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.025354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc09ba-272f-457f-bda3-ce2e44f45738 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.048219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.048850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.049121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.049485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.050608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.052184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.053942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 111/227] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.054697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.064327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.064873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.065152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.065424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.066147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:13.066451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.067886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.077018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 117/228] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.077893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.115511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:13.158001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:13.166141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d130bc-7d63-41f5-b835-ccbd06074178 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.182137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b23b4-7fe6-4150-b5ef-dd25477cf3ef None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.204130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.205048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.205459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.205936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.206790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=95594ec7979d467380701c3d20811b30) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:13.207376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.209172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.214890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 112/229] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:13 2022] GET /identity/v3/role_assignments?user.id=95594ec7979d467380701c3d20811b30&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.215834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.225120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.225679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.225959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.226227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.231693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.233410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.235174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 118/230] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:13 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.241225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.323350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ca16b-7d8a-41d6-985a-7aaabba738b8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.343452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:13.362656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.363684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.364099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.364876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.366651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.368611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.370494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 113/231] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:13 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.377151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.435742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976a298-b877-4fd5-b90b-c90cd3a35127 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.482249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.483654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.484503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.484970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/users/d8f8fbe039fe4790a9606b5c699437c4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.487666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, user_id=d8f8fbe039fe4790a9606b5c699437c4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.489931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.507551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 119/232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:13 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/users/d8f8fbe039fe4790a9606b5c699437c4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:53:13.577822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b07fe-0cd3-4e29-91da-080e933ea03e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.635980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.636654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.636932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.637206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.638535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.640064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085236c-a96d-41ac-86eb-a6d444ca86a0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.642474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 114/233] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:13 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:13.649409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:13.656357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:13.790528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8eef10-4dfc-4a38-8712-39a7ba1588b7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:13.825159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:13.825821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:13.826096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:13.826366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/users/d8f8fbe039fe4790a9606b5c699437c4/roles/ee67389746c245038655ca9ddc941126` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:13.828637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, user_id=d8f8fbe039fe4790a9606b5c699437c4, role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:13.830034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b20a7e-31d7-468a-9384-a1ff4babbad0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:13.846759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 120/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:13 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/users/d8f8fbe039fe4790a9606b5c699437c4/roles/ee67389746c245038655ca9ddc941126 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:14.525233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a427-f80a-43cd-bfc8-328fe0da4a55 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:14.525748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a427-f80a-43cd-bfc8-328fe0da4a55 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:14.525748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4a427-f80a-43cd-bfc8-328fe0da4a55 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:14.527284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 115/235] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:14.543629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a39f2-ac6d-4c2d-9525-4db497775521 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:14.544080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a39f2-ac6d-4c2d-9525-4db497775521 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:14.544080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a39f2-ac6d-4c2d-9525-4db497775521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:14.567173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5e4a39f2-ac6d-4c2d-9525-4db497775521 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:14.568473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5e4a39f2-ac6d-4c2d-9525-4db497775521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:14.646368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:14.660042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 121/236] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:14.748048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf15e8-8dd1-480a-8251-4589f712a006 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:14.748367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf15e8-8dd1-480a-8251-4589f712a006 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:14.748661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf15e8-8dd1-480a-8251-4589f712a006 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:14.750362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 116/237] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:14.755599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:14.756101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:14.756567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:14.782257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:14.783828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:14.868683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 122/238] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:15.056248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf15e8-8dd1-480a-8251-4589f712a006 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.127550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:15.176300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf15e8-8dd1-480a-8251-4589f712a006 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.211367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.211997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.212275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.212547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.223300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:15.228164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:15.229057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:15.229851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 117/239] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:15.232433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.240889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.401772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee527e5-4876-49ac-8c54-c28ce31aa76a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.405177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a9925-6787-49f5-9b56-5b8edb500d6e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.428416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:15.446462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:15.457549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.458285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.458632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.458970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.460194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:15.461716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:15.463404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 123/240] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:15.469553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.470484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.471125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.471405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.471670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.472339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:15.472638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:15.474033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:15.499166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 118/241] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:15.513257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.616370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847112cf-39bb-4db9-9e4b-dd3d333dc710 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.662103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.662782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.664234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.664525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.665189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:15.666658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703119f-2634-4b53-99f2-c8c68b4df3b7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:15.701417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a04a-af9d-46ca-b08b-143e5b7a29b2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.710698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 124/242] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:15 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:53:15.713152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6b53-2af2-47d6-9527-659aed68d390 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.713700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6b53-2af2-47d6-9527-659aed68d390 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.714179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c6b53-2af2-47d6-9527-659aed68d390 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.717081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 125/243] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:15.723473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6afc-01cc-41c9-91c8-3cc77e5e0e76 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.723795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6afc-01cc-41c9-91c8-3cc77e5e0e76 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.724021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6afc-01cc-41c9-91c8-3cc77e5e0e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.743815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-49bd6afc-01cc-41c9-91c8-3cc77e5e0e76 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:15.745079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-49bd6afc-01cc-41c9-91c8-3cc77e5e0e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:15.751817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.752882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.753404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.753924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] PATH_INFO: `/v3/users/8397860727794f8d950889918a253b9f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.774606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] RBAC: Authorizing `identity:get_user(user_id=8397860727794f8d950889918a253b9f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:15.779340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:15.782610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 119/244] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:15 2022] GET /identity/v3/users/8397860727794f8d950889918a253b9f => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:15.791646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:15.808265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:15.855633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 126/245] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:15.866263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:15.938051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada60b0-b397-43ca-8e6d-e685d7d0d97a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.938534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada60b0-b397-43ca-8e6d-e685d7d0d97a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.938838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada60b0-b397-43ca-8e6d-e685d7d0d97a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.940472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 127/246] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:15.947005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a9b9-8868-4272-854a-1b510529028e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.947323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a9b9-8868-4272-854a-1b510529028e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.947638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a9b9-8868-4272-854a-1b510529028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:15.957573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954704-44af-44ef-8ec5-ae7f1ae78f9e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:15.975863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f779a9b9-8868-4272-854a-1b510529028e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:15.977368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f779a9b9-8868-4272-854a-1b510529028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:15.995072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:15.995688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:15.995964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:15.996540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.064148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 128/247] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:16.084387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:16.085870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:16.117495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:16.180128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779a9b9-8868-4272-854a-1b510529028e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.191522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 120/248] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:16.197125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.295070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779a9b9-8868-4272-854a-1b510529028e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.300418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a20512-eeae-4741-acbc-b0f3782141af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.300872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:16.333142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:16.334031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.334881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.335102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.335541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.338988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.339812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.340226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.340653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.341976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:16.344992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:16.346887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 121/249] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:16.351426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:16.353038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cf6965f8-524a-44d1-8ca1-d1ec72c7ea6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:16.357769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:16.358831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:16.360238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 129/250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:16.366828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff455-cd43-4659-b4e0-716b26124fae None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.367158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff455-cd43-4659-b4e0-716b26124fae None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.367431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff455-cd43-4659-b4e0-716b26124fae None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.368953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 130/251] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:16.369659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff455-cd43-4659-b4e0-716b26124fae None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.475907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f86873-4810-4e56-bf23-f40f7ab7415a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.495820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff455-cd43-4659-b4e0-716b26124fae None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.523052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.524308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.525179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.525908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.527146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=8397860727794f8d950889918a253b9f) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:16.528065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:16.530354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:16.536205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.536811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.537086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.537354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.538017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:16.538312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:16.538997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 122/252] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/role_assignments?user.id=8397860727794f8d950889918a253b9f&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=a856bfff6600429a90907865739b465b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:16.539786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321550e-e80f-4cd4-87ef-98ed8f8649b5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:16.540330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.552291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 131/253] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:16.559294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.559591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.559864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.586654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:16.586654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:16.637960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:16.667347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:16.709365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 132/254] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:16.710230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.773097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6745d4a-719a-4251-9d79-c80004b0b93e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.859812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.860451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.860741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.860982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.866331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852537fd-664e-4987-ac72-5380a0d4eba6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:16.891957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:16.897353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:16.898229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:16.899010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 123/255] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:16.912019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.923717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:16.924357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:16.924633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:16.924902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] PATH_INFO: `/v3/users/95594ec7979d467380701c3d20811b30` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:16.926541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] RBAC: Authorizing `identity:get_user(user_id=95594ec7979d467380701c3d20811b30)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:16.927969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:16.929859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 133/256] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/users/95594ec7979d467380701c3d20811b30 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:16.934913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:16.966826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:17.113857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77411935-c693-438b-98a1-622af4e33a8b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.129974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd10c0e-fcb9-49a9-9701-f0c00c7992a0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:17.189769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.190416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.190684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.190990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.192947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:17.194519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:17.196391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 134/257] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:17.196985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.206755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.207383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.207659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.207919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.208588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:17.208875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:17.210285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:17.226160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 124/258] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:17.226975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.387893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:17.398071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db04d3-e27c-424e-a32c-3b8678561a4c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.411679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186cd94-3abc-4c9c-a585-0f2a344555d6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.459053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.459680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.459949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.460210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.481038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:17.487321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:17.488646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 135/259] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:17.489790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.490616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.492123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.492123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.492123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/users/95594ec7979d467380701c3d20811b30/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.496062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, user_id=95594ec7979d467380701c3d20811b30, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:17.497815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:17.518727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 125/260] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:17 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/users/95594ec7979d467380701c3d20811b30/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:17.519358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.620493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:17.692885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea803d55-5bf0-4e82-ab48-40fac24783e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.736462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012c2f5-d5ef-4f15-8a8f-8d07bf330537 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.755634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.756273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.756546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.756874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.758023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:17.759572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:17.761159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 136/261] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:17 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:17.769007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.799129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:17.823777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7147383-3513-4312-842e-2612af3f822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.824451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7147383-3513-4312-842e-2612af3f822c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.824685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7147383-3513-4312-842e-2612af3f822c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.824960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7147383-3513-4312-842e-2612af3f822c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:17.825644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7147383-3513-4312-842e-2612af3f822c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:17.825940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7147383-3513-4312-842e-2612af3f822c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:17.827373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7147383-3513-4312-842e-2612af3f822c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:17.839289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 126/262] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:17.844963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7147383-3513-4312-842e-2612af3f822c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:17.884743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b8975-052b-405c-a650-9258be5dd419 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:17.925038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:17.926538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:17.927116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:17.927823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.002105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.004090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.030412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7147383-3513-4312-842e-2612af3f822c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:18.035368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.051053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.100868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:18.101524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.101802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.102075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] PATH_INFO: `/v3/users/d8f8fbe039fe4790a9606b5c699437c4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.107877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] RBAC: Authorizing `identity:get_user(user_id=d8f8fbe039fe4790a9606b5c699437c4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.109622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.111867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 127/263] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:17 2022] GET /identity/v3/users/d8f8fbe039fe4790a9606b5c699437c4 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:18.119111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 137/264] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 03:53:18.119932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:18.124833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:18.224613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eb48-9aff-441f-9303-7859f4593a45 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:18.259757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:18.260311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.260590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.260863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.262054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.263617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.265269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 138/265] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:18 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:18.265762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.277134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:18.295306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52056267-99de-4c11-be67-cf035f79f73b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:18.361506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:18.362130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.362410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.362676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.438123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.488681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.490508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.509996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278202b-293f-427c-84c9-ba5cba801e08 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:18.556549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:18.557162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.557438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.557761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.558755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=d8f8fbe039fe4790a9606b5c699437c4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:18.559086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.560526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4512e7d-2cee-4dfc-b5b2-fede81f96dcd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.569122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 139/266] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:18 2022] GET /identity/v3/role_assignments?user.id=d8f8fbe039fe4790a9606b5c699437c4&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:18.570364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bddb06-2129-44fa-9817-0b8897e46066 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.570640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bddb06-2129-44fa-9817-0b8897e46066 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.570908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bddb06-2129-44fa-9817-0b8897e46066 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.576653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 140/267] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:18.591214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.591821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.592301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.631851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:18.635019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:18.644566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 128/268] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 03:53:18.652153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:18.740201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.785837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 141/269] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:18.817904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:18.848309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6077e-865d-4621-99e1-42bd4d31439f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:18.890625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:18.891282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:18.891559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:18.891827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:18.893050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:18.894499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:18.896477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 129/270] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:18 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:18.902946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.017256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360d1ad-7b15-4521-bafa-62a6cbf5bd86 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.054222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.055180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.055647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.056116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.057042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=d8f8fbe039fe4790a9606b5c699437c4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:19.057587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:19.059302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb828f0-0aa3-404f-bc60-c30a9f065737 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:19.066299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 142/271] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:18 2022] GET /identity/v3/role_assignments?user.id=d8f8fbe039fe4790a9606b5c699437c4&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=ee67389746c245038655ca9ddc941126 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:19.093476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.240625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a693c5-20fa-4c7c-bbb5-945ff414aeb0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.277274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.277987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.278263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.278525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.289781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:19.291518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2afc65-b608-4c7c-ab6c-20ea26fbbfe7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.291800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2afc65-b608-4c7c-ab6c-20ea26fbbfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.292124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2afc65-b608-4c7c-ab6c-20ea26fbbfe7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.293469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 143/272] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:19.294657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:19.295537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:19.296700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 130/273] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:19.299503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15dec-8fc7-47b3-8a85-e87147bac7ba None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.299503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15dec-8fc7-47b3-8a85-e87147bac7ba None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.299829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad15dec-8fc7-47b3-8a85-e87147bac7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.304819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.309130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:19.338960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aad15dec-8fc7-47b3-8a85-e87147bac7ba None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:19.340844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aad15dec-8fc7-47b3-8a85-e87147bac7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:19.397165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:19.418912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 144/274] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:19.424321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890f45-09c0-4101-ba8a-dfd32e8133bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.424595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890f45-09c0-4101-ba8a-dfd32e8133bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.424861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890f45-09c0-4101-ba8a-dfd32e8133bb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.426249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 145/275] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:19.429995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8d4ce-5336-4877-baff-f25d6f4e3349 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.431176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.431577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.431843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.452640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:19.454215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:19.465880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.466958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.467471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.467956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.468912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:19.469457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:19.471181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:19.480768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 131/276] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:19.488254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.526373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 146/277] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:19.568643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:19.610354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1182-a154-42e7-a1a2-d181c76b9d4f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.645906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.646533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.646810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.647109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] PATH_INFO: `/v3/users/39c743fed60341deab5901b717221c9f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.666770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] RBAC: Authorizing `identity:get_user(user_id=39c743fed60341deab5901b717221c9f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:19.669450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:19.672947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 132/278] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/users/39c743fed60341deab5901b717221c9f => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:19.678809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.691860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.802823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df08ff-3bf1-42ea-857b-e3f89fd3b7fd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.814497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:19.819272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa4c34-30b0-4ed8-8b43-fd17d63a2319 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:19.839868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.840475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.840754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.841105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.841742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, user.id=39c743fed60341deab5901b717221c9f, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:19.842045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:19.843605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:19.843759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:19.849515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 147/279] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/role_assignments?user.id=39c743fed60341deab5901b717221c9f&scope.system=all&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:19.855518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:19.856227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:19.856441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:19.856710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:19.857901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:19.859571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:19.860505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:19.861428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 133/280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:19.867196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.003064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e88377-2dd8-4876-9cd7-fde4de98c472 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.006773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46082-b905-4348-aa28-20d251df53f9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.048471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.048692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.049251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.049526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.049791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.050230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.050703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.051202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.052054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.053714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7316d-6d25-4189-b3d6-c06721b4e36f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.060845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:20.067306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:20.068236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:20.069255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 148/281] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:20.074712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.083380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 134/282] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:19 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:20.108197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5e94-f2a3-43f7-90c1-3d95b7405bf2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.108517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5e94-f2a3-43f7-90c1-3d95b7405bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.108778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce5e94-f2a3-43f7-90c1-3d95b7405bf2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.110896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 135/283] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:20.116486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.116809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.117017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.129174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:20.139543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:20.141173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:20.207069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:20.211995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350d135-48af-4f26-a1c3-28130b64cf8d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.243695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 136/284] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:20.257770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.258383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.258659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.258930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.259638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:20.259937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.261312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.279928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 149/285] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:20.281261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.391800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969913-9724-4dc9-ad11-d8e015e449dd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.427234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.427858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.428140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.428407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] PATH_INFO: `/v3/users/8397860727794f8d950889918a253b9f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.429533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] RBAC: Authorizing `identity:get_user(user_id=8397860727794f8d950889918a253b9f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.431079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.433048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 137/286] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/users/8397860727794f8d950889918a253b9f => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:20.438366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.507941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.558704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62456-938f-4d2c-9b74-434862dcbc92 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.566392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:20.568892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:20.604649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.605558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.605840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.606111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.607793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.609731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.612394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 150/287] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:20.617590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.641522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3694-89fe-4d20-83d0-22fe0f3fb288 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.701969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.702664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.703065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.703318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.724336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:20.729252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:20.730126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:20.730915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:20.731549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 138/288] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:20.738232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:20.776782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc162b7-8776-4bbe-a0ef-bd5aa4b39fe0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.820680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.821560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.821946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.822415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/users/8397860727794f8d950889918a253b9f/roles/a856bfff6600429a90907865739b465b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.825750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, user_id=8397860727794f8d950889918a253b9f, role_id=a856bfff6600429a90907865739b465b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.836034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.864921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb35d-2441-41b0-9a76-eb19e806168a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:20.873261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:20.876682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 151/289] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:20 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/users/8397860727794f8d950889918a253b9f/roles/a856bfff6600429a90907865739b465b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:20.926366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:20.926994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:20.927291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:20.927554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:20.928234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:20.928529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:20.929982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:20.947729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 139/290] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:20.954071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:21.004332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:21.102215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84066b80-5dd7-4e9b-87ec-2d3d7d670277 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:21.140129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:21.140844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.141120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.141379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] PATH_INFO: `/v3/users/95594ec7979d467380701c3d20811b30` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.142502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] RBAC: Authorizing `identity:get_user(user_id=95594ec7979d467380701c3d20811b30)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:21.143962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:21.145772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 152/291] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:20 2022] GET /identity/v3/users/95594ec7979d467380701c3d20811b30 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:21.151317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:21.289497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d6151-7e80-45ce-a989-c86ad7b0ffee None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:21.340878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:21.341491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.341767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.342030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.389246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:21.422761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:21.424268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:21.545677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 140/292] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:21.551298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:21.669429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:21.685776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9075-83cc-458d-8b5d-8880235dc7fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:21.730164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:21.730799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.731102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.731380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.732523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:21.733964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cae93-17e3-413b-be5f-e777bf60619e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:21.736661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 153/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:21 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:21.744202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:21.880234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:21.924879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeddfb-0a4a-4c4f-83cc-bd879da4c427 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:21.956938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213e76-1b00-4e74-8c50-743a037c257d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.957882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213e76-1b00-4e74-8c50-743a037c257d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.958378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2213e76-1b00-4e74-8c50-743a037c257d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.960278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 154/294] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:21.965747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e85f6-257d-4c20-ac60-1c29129389d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.966276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e85f6-257d-4c20-ac60-1c29129389d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.966735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e85f6-257d-4c20-ac60-1c29129389d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.978664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:21.979636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:21.982800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:21.983314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:21.984317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=95594ec7979d467380701c3d20811b30) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:21.985105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:21.986904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:21.998152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 141/295] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:21 2022] GET /identity/v3/role_assignments?user.id=95594ec7979d467380701c3d20811b30&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:22.006901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a18e85f6-257d-4c20-ac60-1c29129389d9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:22.006901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a18e85f6-257d-4c20-ac60-1c29129389d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:22.075482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 155/296] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:22.410914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:22.521072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bcb1f-8a66-4010-a994-25e14450ea7c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:22.558490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:22.565274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:22.566215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:22.566683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:22.567185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:22.578837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:22.585961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:22.586832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:22.587679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:22.590283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 142/297] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:22.592602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348addbc-f502-4e46-82e0-0da8ca9852f3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:22.592885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348addbc-f502-4e46-82e0-0da8ca9852f3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:22.593154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348addbc-f502-4e46-82e0-0da8ca9852f3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:22.594610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 156/298] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:22.599271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:22.599779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:22.600150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:22.600433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:22.622982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:22.624518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:22.696760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 157/299] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:22.713375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42682e-845d-4a5b-9aae-6d84055de3a4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:22.752119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:22.752726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:22.753003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:22.753291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:22.753943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:22.754239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:22.755663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:22.764736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 143/300] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:22.770654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:22.794537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:22.876476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34920c67-64b2-4e7a-a246-406c582ec404 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:22.914687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:22.915350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:22.915630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:22.915905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] PATH_INFO: `/v3/users/39c743fed60341deab5901b717221c9f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:22.917051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] RBAC: Authorizing `identity:get_user(user_id=39c743fed60341deab5901b717221c9f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:22.918502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:22.920481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 158/301] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:22 2022] GET /identity/v3/users/39c743fed60341deab5901b717221c9f => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:22.926228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:22.966824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:22.988037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.036378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd329fa-cdc6-4774-a078-b1701d6f3f18 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.060138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.072704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c78f94-26c5-4274-b08d-825972886b41 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.073622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.074344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.074622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.074895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] PATH_INFO: `/v3/system/users/39c743fed60341deab5901b717221c9f/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.076342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=39c743fed60341deab5901b717221c9f, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.077751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16938cac-fbaf-4d70-bdf6-c05af72150b0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.087567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 144/302] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 31 03:53:22 2022] PUT /identity/v3/system/users/39c743fed60341deab5901b717221c9f/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:23.088984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154573-7038-4252-89cb-47ebd0723a56 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.089334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154573-7038-4252-89cb-47ebd0723a56 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.089675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154573-7038-4252-89cb-47ebd0723a56 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.091075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 145/303] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:23.096830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd63f-9110-4609-9b3a-56a399189049 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.097173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd63f-9110-4609-9b3a-56a399189049 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.097560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd63f-9110-4609-9b3a-56a399189049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.118609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a57cd63f-9110-4609-9b3a-56a399189049 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:23.120217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a57cd63f-9110-4609-9b3a-56a399189049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:23.122085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.123203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.123674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.124103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.139445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:23.146617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:23.147692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:23.148476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:23.151070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 159/304] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:23.157325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.207075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 146/305] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:23.207075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4ec8-8014-4c1d-a8ff-cd26fbe795c4 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.207075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4ec8-8014-4c1d-a8ff-cd26fbe795c4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.207075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4ec8-8014-4c1d-a8ff-cd26fbe795c4 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.207075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 147/306] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:23.212354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.212641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.212891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.234313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:23.235568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:23.298207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ea47-98f7-4c1b-a110-8e013b125f0f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.301870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.333324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 148/307] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:23.334931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.335655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.335936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.336203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.336862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:23.337159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.338543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.351202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 160/308] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:23.356372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.460635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321beb6e-c459-4a4c-b4fc-760c2ed631bd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.482075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.495631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.496585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.496859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.497124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] PATH_INFO: `/v3/users/2e68c681d6e9489a8395aa17b609662b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.510815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] RBAC: Authorizing `identity:get_user(user_id=2e68c681d6e9489a8395aa17b609662b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.511416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.512346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.514350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 149/309] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/users/2e68c681d6e9489a8395aa17b609662b => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:23.524055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.569216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.597650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4b41-7739-473b-924c-873d29f044a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.636414 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.637040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.637314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.637581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.639383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.640920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.646911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 161/310] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:23.646911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.651375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d251c6-73f3-48c4-bc72-21468d228595 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.691514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.692119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.692393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.692656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.693405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, user.id=2e68c681d6e9489a8395aa17b609662b, scope.system=all) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:23.693706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.695211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.700436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 150/311] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/role_assignments?user.id=2e68c681d6e9489a8395aa17b609662b&scope.system=all&role.id=a856bfff6600429a90907865739b465b => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:23.708202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.733676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.759355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a89cd-a17a-4104-a935-d6bfe4a16110 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.798207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.799071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.799507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.799873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.817336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:23.824106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:23.825250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:23.826134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:23.829005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 162/312] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:23.837190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:23.844419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:23.847794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9adb9-28e7-44de-963c-8025ae14b1af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.891134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.891762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.892043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.892313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] PATH_INFO: `/v3/groups` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.892948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.894889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:23.921796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 151/313] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 31 03:53:23 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:23.950300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111501a8-e0ca-464e-9546-75d55511a68f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:23.989851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:23.991362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:23.991835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:23.992313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:23.993195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:23.993707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:23.995424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:24.003082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:24.011559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 163/314] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:24.014208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:24.126189 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec146a2e-cd5f-4087-8225-3d2375d9c80a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:24.164382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:24.165405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:24.165884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:24.166348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] PATH_INFO: `/v3/users/8397860727794f8d950889918a253b9f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:24.167817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] RBAC: Authorizing `identity:get_user(user_id=8397860727794f8d950889918a253b9f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:24.169624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:24.171960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 152/315] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:24 2022] GET /identity/v3/users/8397860727794f8d950889918a253b9f => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:24.177838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:24.286768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa3f08-0721-4a49-9921-bd1a22b4302b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:24.322099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:24.322708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:24.322985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:24.323274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:24.392260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:24.395221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:24.396687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:24.497063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 164/316] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:24.502431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:24.535911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:24.614800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537b7a5-92ca-49fb-94aa-068659f82492 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:24.650458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:24.651364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:24.651785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:24.652199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:24.653494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:24.655122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793199d-dbda-4ebc-a8c9-efb901d5bb35 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:24.657385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 153/317] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:24 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:24.663358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:24.768653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:24.771803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c412d-d50a-4bc1-9ea9-c560b14ac3ed None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:24.807120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:24.807665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:24.807940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:24.808207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:24.808937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, scope.project.id=46caf8d466c343af892d0910efcfe30e, user.id=8397860727794f8d950889918a253b9f) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:24.809243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:24.810704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156a83a-8329-4629-8bd5-8fa412a22cf9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:24.818343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 165/318] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:24 2022] GET /identity/v3/role_assignments?user.id=8397860727794f8d950889918a253b9f&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=a856bfff6600429a90907865739b465b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:25.141116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3a8-f0a6-4949-8ebb-94f34cf7fa18 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.141941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3a8-f0a6-4949-8ebb-94f34cf7fa18 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.142449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3a8-f0a6-4949-8ebb-94f34cf7fa18 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.144250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 154/319] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:25.149390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.149778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.149935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.168794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:25.169981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:25.243528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 166/320] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:25.506805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83f3a8-f0a6-4949-8ebb-94f34cf7fa18 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:25.612713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:25.620781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83f3a8-f0a6-4949-8ebb-94f34cf7fa18 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:25.655874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:25.657108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.657377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.657644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.668657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6eff404f-1529-45c6-9278-a096f0b24f21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:25.674110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:25.674963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:25.680169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 155/321] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:25.681379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:25.692384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ffe40-c8e0-4b1f-a531-8821e98585c1 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.692670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ffe40-c8e0-4b1f-a531-8821e98585c1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.692997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ffe40-c8e0-4b1f-a531-8821e98585c1 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.694926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 156/322] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:25.699899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.700173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.700461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.720386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:25.721577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:25.782983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:25.789883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929786e-1b15-4c6c-ab1f-9831fbd594dc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:25.796338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 157/323] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:25.826171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:25.826778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.827050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.827348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.828213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:25.828625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:25.830009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393720-55ab-4dbe-bcef-b06a501432c8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:25.842813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 167/324] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:25.844595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:25.904182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c72ff0-2d86-4249-9df7-b3af215e45c5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.904511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c72ff0-2d86-4249-9df7-b3af215e45c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.904737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c72ff0-2d86-4249-9df7-b3af215e45c5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.906346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 168/325] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:25.911107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:25.911353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:25.911619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:25.930301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:25.931541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:25.948836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:25.966481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381570-3533-44ec-bffe-01f2f3bc6900 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.002695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.003360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.003637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.003890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] PATH_INFO: `/v3/users/39c743fed60341deab5901b717221c9f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.005023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] RBAC: Authorizing `identity:get_user(user_id=39c743fed60341deab5901b717221c9f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.006498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.007206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 169/326] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:26.013098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 158/327] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:25 2022] GET /identity/v3/users/39c743fed60341deab5901b717221c9f => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:26.014023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.073333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.100632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:26.123408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9965ced-10ae-45f3-9802-9bf98bb31311 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.162744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.163419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.163632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.163895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.164625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] RBAC: Adding query filter params (role.id=27e940be03fb44c8974bdff2bc53de9b, user.id=39c743fed60341deab5901b717221c9f, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:26.164931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.166377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.178142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 170/328] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 31 03:53:26 2022] GET /identity/v3/role_assignments?user.id=39c743fed60341deab5901b717221c9f&scope.system=all&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:26.180606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81254c-1665-4187-8896-0de75e28ace3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.220220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.221167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.221614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.222079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.235590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:26.240962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:26.241886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:26.242672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:26.244622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 159/329] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:26.251179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.284661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.302073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:26.378074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c4818-49b3-45de-96eb-aa4d065b1971 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.399484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a3c2-1321-493d-a8e9-db9d567015a7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.411126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.411734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.412010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.412280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.412939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:26.413244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.414610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.422616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 171/330] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:26.428358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.432834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.433458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.433730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.433995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.435104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.436613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.437002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:26.438176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 160/331] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:26.443904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.524569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:26.528975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02697b-5947-4b83-a0e8-5c54101597a3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.543158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164695-df42-450f-a79c-06519f6d9f38 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.562939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.563605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.563879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.564150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] PATH_INFO: `/v3/users/2e68c681d6e9489a8395aa17b609662b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.565330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] RBAC: Authorizing `identity:get_user(user_id=2e68c681d6e9489a8395aa17b609662b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.566780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.568683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 172/332] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:26 2022] GET /identity/v3/users/2e68c681d6e9489a8395aa17b609662b => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:26.574068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:26.576638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.577251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.577547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.577857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] PATH_INFO: `/v3/groups` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.578479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.579971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea319f4-60b3-4612-bdb0-c004eade8c94 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.595776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 161/333] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 31 03:53:26 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:26.658270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:26.685344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cfb54-f8b7-4726-af39-315d2cd4ade0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:26.725733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:26.726391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:26.726655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:26.726932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] PATH_INFO: `/v3/system/users/2e68c681d6e9489a8395aa17b609662b/roles/a856bfff6600429a90907865739b465b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:26.728374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2e68c681d6e9489a8395aa17b609662b, role_id=a856bfff6600429a90907865739b465b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:26.729768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dc7d6-d6ad-4318-bc81-80d52a6aaa11 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:26.739290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 173/334] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 31 03:53:26 2022] PUT /identity/v3/system/users/2e68c681d6e9489a8395aa17b609662b/roles/a856bfff6600429a90907865739b465b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:53:28.691624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701194-3784-4e3a-827c-e9a95f41eaef None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:28.692214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701194-3784-4e3a-827c-e9a95f41eaef None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:28.692214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf701194-3784-4e3a-827c-e9a95f41eaef None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:28.693806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 162/335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:28.699171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325a8b6-3918-4d02-89f8-0b33fc6d9501 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:28.699617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325a8b6-3918-4d02-89f8-0b33fc6d9501 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:28.699816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325a8b6-3918-4d02-89f8-0b33fc6d9501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:28.724102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7325a8b6-3918-4d02-89f8-0b33fc6d9501 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:28.725301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7325a8b6-3918-4d02-89f8-0b33fc6d9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:28.807893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 174/336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:28.857443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858ec9-b34a-45ef-a292-7a6453b5b1da None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:28.857748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858ec9-b34a-45ef-a292-7a6453b5b1da None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:28.857987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858ec9-b34a-45ef-a292-7a6453b5b1da None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:28.859920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 163/337] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:28.865210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a960-a060-4062-92b5-39bdeb20579b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:28.865752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a960-a060-4062-92b5-39bdeb20579b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:28.866222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a960-a060-4062-92b5-39bdeb20579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:28.890405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ca34a960-a060-4062-92b5-39bdeb20579b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:28.892036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca34a960-a060-4062-92b5-39bdeb20579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:28.973022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:28.979788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 175/338] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:29.232750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ad0b-7fb2-4ecc-a59d-c5ebc20433e6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.232931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ad0b-7fb2-4ecc-a59d-c5ebc20433e6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.233196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ad0b-7fb2-4ecc-a59d-c5ebc20433e6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.235240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 164/339] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:29.248904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4ec61-7597-45d6-8233-b0cee758ec46 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.249198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4ec61-7597-45d6-8233-b0cee758ec46 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.249474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4ec61-7597-45d6-8233-b0cee758ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.276052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-94c4ec61-7597-45d6-8233-b0cee758ec46 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:29.277253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94c4ec61-7597-45d6-8233-b0cee758ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:29.307527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ad0b-7fb2-4ecc-a59d-c5ebc20433e6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:29.392818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 176/340] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:29.419670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8eea5-c973-4d72-8751-5535cb34f882 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.420277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8eea5-c973-4d72-8751-5535cb34f882 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.420562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8eea5-c973-4d72-8751-5535cb34f882 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.422406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 177/341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:29.429101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.429387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.429662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.459934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:29.461110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:29.515727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ad0b-7fb2-4ecc-a59d-c5ebc20433e6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:29.560324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 178/342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:29.560949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:29.581784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:29.582416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.582689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.582950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.605911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:29.612117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:29.613052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 03:53:29.613837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:29.620294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 165/343] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:29.622165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:29.641379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:29.643262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:29.734869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07713fdb-6b1b-479b-9462-6bb7e0b2ffb7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:29.758330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fb81-d325-4418-b171-2d78840bf325 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:29.813431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:29.814287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:29.814625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.814919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:29.815220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.815487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.815736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:29.816190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:29.817048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:29.817770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:29.819391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:29.833743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 166/344] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:29.838623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:29.847979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:29.852902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:29.853762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:29.854985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 179/345] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:29.855692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:29.927354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:29.977757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d84e-2b1f-4745-9066-0b99bbd2bcd7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.021069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.022059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.022578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.023207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] PATH_INFO: `/v3/users/2e68c681d6e9489a8395aa17b609662b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.024547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] RBAC: Authorizing `identity:get_user(user_id=2e68c681d6e9489a8395aa17b609662b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.026614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.028944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 167/346] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:53:29 2022] GET /identity/v3/users/2e68c681d6e9489a8395aa17b609662b => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:30.030130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.049742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbdc8-31a3-457e-ac76-31a34000e108 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.067674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.089988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.090977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.091585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.092048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.105600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:30.111383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:30.112288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:30.113176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:30.115591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 180/347] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:30.116911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.145872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c889adc-ca00-4a2c-9c59-b8a1926e5a67 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.176832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.181758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.182368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.182643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.182909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.183608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:30.183907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.185295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.195071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 168/348] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:30.196019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.233756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ef3d-005b-4912-aed4-9979912224f5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.277617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.278679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.279209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.279673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.296806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:30.302503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:30.303410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:30.304194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:30.305314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1224c-b21e-4227-b04f-264ce430453c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.306647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 181/349] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:30.307751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.354409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.354941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.355254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.355525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.356258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] RBAC: Adding query filter params (role.id=a856bfff6600429a90907865739b465b, user.id=2e68c681d6e9489a8395aa17b609662b, scope.system=all) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:30.356562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.358022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.358672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.371799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 169/350] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/role_assignments?user.id=2e68c681d6e9489a8395aa17b609662b&scope.system=all&role.id=a856bfff6600429a90907865739b465b => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:30.372663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.433475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae497eb-4693-40c7-9f1e-3443664b473e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.435418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.475474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.476507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.476981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.477442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.478315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:30.478835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.480511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.490676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b05eb4-e98c-47ed-b7e0-9d895e4f39bf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.491917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 182/351] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:30.493108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.532513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.533135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.533415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.533684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.615623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.622184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039fade-ec16-4b7d-8dda-98b7184ae2ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.628175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.629683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.666907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.668239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.668745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.669357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.670324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:30.670995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.673579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:30.686797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 183/352] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:30.688163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.704793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.770853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 170/353] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:30.772391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:30.829944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92377974-f5ea-4687-ac3f-cde54f71cf1e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.871824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.872540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.872820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.873090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.883754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8068-6253-4b31-8a84-6ab4a372a97f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:30.913163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.930760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:30.931497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:30.931777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:30.932047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:30.958259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:30.972483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:30.974880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.016784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.018272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd6590-f3cf-43b7-a763-7e4f290f4181 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.098587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 184/354] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:31.104152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.138186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 171/355] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdc1f8-00e2-440c-a4a1-9b600ed22481 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdc1f8-00e2-440c-a4a1-9b600ed22481 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdc1f8-00e2-440c-a4a1-9b600ed22481 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 172/356] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a426d-18aa-45f4-86a0-35ae27aaa6f3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.143092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a426d-18aa-45f4-86a0-35ae27aaa6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.143595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a426d-18aa-45f4-86a0-35ae27aaa6f3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.147980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 173/357] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:31.148738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a426d-18aa-45f4-86a0-35ae27aaa6f3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.180461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.217068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.228071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a82357-27b6-41fd-9981-3ec6b9165621 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.269848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.270858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.271405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.271860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.273200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.274709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.277575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a426d-18aa-45f4-86a0-35ae27aaa6f3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.277717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 185/358] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.278394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.318192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.318827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.319207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.319481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.320596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.322039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353cdcd-2b16-4e7c-ab5b-9766133464ab None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.324020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 174/359] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.324978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0773f-0e8f-4732-944e-0b78fa067df5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.325252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0773f-0e8f-4732-944e-0b78fa067df5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.325516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0773f-0e8f-4732-944e-0b78fa067df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.345224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f5e0773f-0e8f-4732-944e-0b78fa067df5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:31.346438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5e0773f-0e8f-4732-944e-0b78fa067df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:31.402566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5b70e-e114-48f1-ac42-9be904ced854 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.426019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 175/360] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:31.427684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.428069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.428758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.436377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.442473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.443173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.443426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.443747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.444914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.446367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.450373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 186/361] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.450373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.453998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:31.456144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:31.515647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.541018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 176/362] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:31.542130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.579964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bc6d6-a34f-4293-b476-50036f23fc31 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.626753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.627524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.627869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.628265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.636455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.638091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.640591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 187/363] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.641647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.670492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e801d5-686b-40ca-8da7-3ff89f90b979 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.720006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.720642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.720920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.721188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.722308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.723785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.725530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 177/364] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.726704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.730785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.771548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7f8a-a165-4665-bed3-21a84ab2443d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.791019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.814401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.815073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.815345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.815617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.816721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.818159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.820097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 188/365] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.820826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.849487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d9c3b-17c1-42d0-ae43-723c9bf0bc47 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.892547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.893171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.893454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.893724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:31.894461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=f2239116cdae485a95fef3677e13e435) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:31.894789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:31.896249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:31.904025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 178/366] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:31.904871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:31.944886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7cc8c-3b36-4401-b224-4793288eb268 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:31.985385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:31.994268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:31.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:31.995251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:31.995527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.009218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:32.014848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:32.015850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:32.016659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:32.018137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 189/367] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:32.019100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.051504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0df3e6-3e4a-4b57-9446-256338200561 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.067694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.094206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.094987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.095608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.095608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.096317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=ee67389746c245038655ca9ddc941126, scope.project.id=f2239116cdae485a95fef3677e13e435) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:32.096627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:32.098080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:32.111056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 179/368] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:31 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=ee67389746c245038655ca9ddc941126 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:32.111056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.170490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09ed73-7ca2-4c69-a236-8647b46f3d55 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.216493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.217026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.217307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.217579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.218797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=46caf8d466c343af892d0910efcfe30e) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:32.219218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:32.220842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:32.231456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 190/369] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:32.232437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.240887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7327e88-37e1-42be-9dca-1ee5ec83f1b1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.290113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.290787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.291102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.291375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.314790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:32.322430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:32.323446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:32.324216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 180/370] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:32.331094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.375971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.423678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.432790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f732c0-802f-476e-9db8-3170f3f059eb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.483941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.484666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.485099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.485380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.486060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:32.487224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:32.488711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:32.500542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 191/371] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:32.510275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.561891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72a99-d909-46c5-9927-708aadbb1f3b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.632042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776523-cc71-4f02-bfea-95ffce14b65f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.650207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.652420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.653164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.653434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.653697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.654412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:32.654707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:32.656331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:32.671098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.672057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.672513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.672792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:32.677452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 181/372] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:32.682781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.756267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:32.757657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:32.876628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 192/373] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:32.881859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:32.913011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.921864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:32.925835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa46a3-1882-4c9c-8110-46f75af0048a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:32.987269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:32.987886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:32.988161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:32.988429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.049022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b932-5331-4640-9ef0-0d13e7b5fc1a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.091951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.092610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.092891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.093196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.095828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.097294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.098962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 193/374] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:33.104727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:33.139947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.142646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.283396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:33.311462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7124c3-7783-4e21-b2c0-542416159a93 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.346024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:33.355889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 182/375] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:33.361722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:33.382936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.383598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.383878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.384150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.387709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.389169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.390986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 194/376] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:33 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:33.398594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:33.503127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f751-be37-4189-b36a-e895c1db6ea3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.539444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.540094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.540356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.540630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] PATH_INFO: `/v3/projects/a624bee162274d80bf52fd9ee90c7807` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.541753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] RBAC: Authorizing `identity:get_project(project_id=a624bee162274d80bf52fd9ee90c7807)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.543226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.545063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 183/377] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:33 2022] GET /identity/v3/projects/a624bee162274d80bf52fd9ee90c7807 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:33.552706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:33.589863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba8fd5-1e88-49e4-84be-56b76b9aad8b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.630387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.630991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.631290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.631554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.632276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=ee67389746c245038655ca9ddc941126, scope.project.id=46caf8d466c343af892d0910efcfe30e) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:33.632577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.634004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.636118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:33.640940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 195/378] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:33 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=ee67389746c245038655ca9ddc941126 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:33.677331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bae3a-f745-4cc4-a2f3-ee70dca5de06 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.715898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.716564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.716844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.717117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] PATH_INFO: `/v3/groups/817b80951c0e43509350a88eac4496f0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.724270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] RBAC: Authorizing `identity:get_group(group_id=817b80951c0e43509350a88eac4496f0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.725784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385355e-1d3e-4271-ba09-fbc5ec8aa168 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.727940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 184/379] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:33 2022] GET /identity/v3/groups/817b80951c0e43509350a88eac4496f0 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:33.733908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:33.767936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:33.871948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59848ec-98fa-4317-a3c3-a9042439eacf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:33.909611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:33.910223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:33.910501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:33.910774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:33.911553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] RBAC: Adding query filter params (group.id=817b80951c0e43509350a88eac4496f0, role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=a624bee162274d80bf52fd9ee90c7807) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:33.911862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:33.913367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09987a0-b0c6-49ee-9a8d-c424b5342304 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:33.920294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 196/380] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:33 2022] GET /identity/v3/role_assignments?group.id=817b80951c0e43509350a88eac4496f0&scope.project.id=a624bee162274d80bf52fd9ee90c7807&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:34.303863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a68cf-b6fb-45dd-bc1d-f8986a5e21c5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.304863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a68cf-b6fb-45dd-bc1d-f8986a5e21c5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.305336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a68cf-b6fb-45dd-bc1d-f8986a5e21c5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.306896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 185/381] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:34.316731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.317646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.318115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.348106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:34.349674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:34.367300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390d7-d888-4626-a288-a36818f9fde6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.367611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390d7-d888-4626-a288-a36818f9fde6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.367849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390d7-d888-4626-a288-a36818f9fde6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.369451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 186/382] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:53:34.375018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.375320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.375787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.411310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:34.412560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:34.460489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 197/383] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:34.503084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 187/384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:34.778601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:34.791761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:34.828577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:34.903806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011ed42-0f0f-4911-a66d-908413624911 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:34.925569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:34.947435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:34.948160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.948507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.948853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.962745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4df0b8ba-1323-4cf0-847c-b7b7dc949edf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:34.970195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:34.971342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:34.972324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:34.973301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 198/385] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:34.974627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99874078-0d30-48b6-b6e7-81aa86801b57 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:34.974977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99874078-0d30-48b6-b6e7-81aa86801b57 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:34.975654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99874078-0d30-48b6-b6e7-81aa86801b57 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:34.981418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 199/386] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:34.981418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99874078-0d30-48b6-b6e7-81aa86801b57 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:34.984455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489862-1b74-4c20-87e3-f50546b262e4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.034704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.035655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.036097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.036504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.050420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f642dc1f-7322-4c87-b568-6d2f5b8d04a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:35.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:35.058080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:35.058860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:35.061970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 188/387] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:35.063631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.064063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.064459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.087754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.095407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:35.097615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:35.127318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99874078-0d30-48b6-b6e7-81aa86801b57 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.169064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.170045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.171015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.171582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.172700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:35.173425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.175701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.191050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 189/388] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:35.191050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.194494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 200/389] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:35.200008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.322591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ba66-76d5-4b20-85b9-58c6d0baac47 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.356383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.357566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.364541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.371131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.371131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.371131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.373019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.374738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.379704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 201/390] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:35.383575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.437322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eba61f-1bb2-4eb8-bc43-f7db56b5ee1a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.485899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.487048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.487539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.488120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.488949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:35.489683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.491289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.508697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 190/391] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:35.508697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.554152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b256e6-4f93-46c4-89b4-7e2d94333f93 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.589577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.590383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.590855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.591248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.592637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.594323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.601111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 202/392] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:35.602126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.632323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b42d4d-5075-46d0-9f21-98a9dee63199 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.649514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.676032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.676285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.677587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.678007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.678481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.700019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:35.707114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:35.708224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:35.709196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:35.711741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 191/393] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:35.712641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.723313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707526c-7453-4520-83cd-06d81bb99a73 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.759790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.760564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.760937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.761303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.762516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.764142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.766405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 203/394] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:35.770976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.849629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c67e7e-e486-4e7a-b4b3-fbb78592e382 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.878521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3b60-ef19-4100-95b5-0849b3e79b47 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:35.891446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.892247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.892621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.892987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/ee67389746c245038655ca9ddc941126` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.895288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, group_id=f950ea0bee09456eb6416d40f58f2ab2, role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.896944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.915011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 192/395] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 31 03:53:35 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/ee67389746c245038655ca9ddc941126 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:35.916443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:35.928146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:35.929022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:35.929467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:35.929863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:35.930687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:35.931156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:35.932678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:35.941144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:35.956256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 204/396] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:35.964485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.005176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.089830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a01158-13c5-432c-9506-dced04bfdd0f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.094271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a3b6-d98c-4548-ba58-5618a0e3e10e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.128846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.129031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.129235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.129482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.131478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.132945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd1300-cadc-47d9-92ac-c2c428e6e5e0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.134497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.134872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 193/397] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:36.135166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.135442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.135709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.136007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfb6f-4e5d-4618-87b4-442566a8f770 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.136285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfb6f-4e5d-4618-87b4-442566a8f770 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.136555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedfb6f-4e5d-4618-87b4-442566a8f770 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.137817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 194/398] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:36.141514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.143165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedfb6f-4e5d-4618-87b4-442566a8f770 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.143269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ff3d-1aaf-4505-9f41-da8848701922 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.145056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 205/399] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:35 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:36.146075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.146349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.146789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.169566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:36.170861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:36.238878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.247704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 206/400] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:36.248596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.274708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.282796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedfb6f-4e5d-4618-87b4-442566a8f770 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.324752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.325910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.326385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.326883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.329154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2239116cdae485a95fef3677e13e435, group_id=f950ea0bee09456eb6416d40f58f2ab2, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.330862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe92b1d-a340-41c3-ae93-b966bf81cb84 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.351805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 195/401] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 31 03:53:36 2022] PUT /identity/v3/projects/f2239116cdae485a95fef3677e13e435/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:53:36.353039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3338c2-57c3-4cc8-9dd6-ef2a7bc35189 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.353318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3338c2-57c3-4cc8-9dd6-ef2a7bc35189 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.353593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3338c2-57c3-4cc8-9dd6-ef2a7bc35189 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.358881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 196/402] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:36.365365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.365365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.365365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.398999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94697101-a2ec-4b52-ad06-91e48d4d86c5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.416108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:36.417315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:36.457560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.458627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.459189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.459658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.462003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.464074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.466297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 207/403] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:36 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:36.473767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.555145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 197/404] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:36.556078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.567619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.607315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b09b-8e3f-4173-adb2-2e9eb2c6c683 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.642771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.643511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.643789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.644112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.646000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, group_id=f950ea0bee09456eb6416d40f58f2ab2, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.647534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.663845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 208/405] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 31 03:53:36 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:36.674336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.686968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520b9d9-0bb2-4807-a82a-45918786b0dc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.728183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.728867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.729147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.729414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.741883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:36.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:36.748320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:36.749095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 198/406] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:36.754849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.861953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10873ce-4934-4925-9a83-e50860e747ad None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:36.892513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:36.899678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:36.900294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:36.900914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:36.901737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:36.902531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:36.902901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:36.905100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:36.906605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:36.916102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 209/407] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:36.921637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.019736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a742f2-94f5-4c09-bf73-fb24daac6a15 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.027098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d45e0-9bdc-4472-98d4-4a3af04f8eb8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.061957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.062755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.063161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.063530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.064939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.066481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.068583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 210/408] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:36 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:37.074003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.080166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.081000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.081439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.081853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.106413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:37.111862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:37.112737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:37.113503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:37.116185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 199/409] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:37.122290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.133736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:37.137361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:37.180921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64316f2-7f30-4efc-a938-adf3e4faa422 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.216744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.217382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.217672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.217930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.219186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.220628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.222688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 211/410] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:37 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:37.228327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.240698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b797fa-87ad-4b33-beb0-e1c6e3bcdd22 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.287010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.288377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.288972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.289445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.290293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:37.290771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.292580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.302422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 200/411] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:37.312207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.331271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07e68-bc95-40de-9907-e18d759bd989 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.359552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:37.366787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.367821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.368423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.369888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/ee67389746c245038655ca9ddc941126` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.373256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] RBAC: Authorizing `identity:create_grant(project_id=46caf8d466c343af892d0910efcfe30e, group_id=f950ea0bee09456eb6416d40f58f2ab2, role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.374866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.399362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 212/412] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 31 03:53:37 2022] PUT /identity/v3/projects/46caf8d466c343af892d0910efcfe30e/groups/f950ea0bee09456eb6416d40f58f2ab2/roles/ee67389746c245038655ca9ddc941126 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:37.435513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:37.468220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1518-a940-4c69-a2fe-fbb67a109119 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.513692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.514812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.515368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.515942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] PATH_INFO: `/v3/groups/817b80951c0e43509350a88eac4496f0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.517471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] RBAC: Authorizing `identity:get_group(group_id=817b80951c0e43509350a88eac4496f0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.519299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.521633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 201/413] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:37 2022] GET /identity/v3/groups/817b80951c0e43509350a88eac4496f0 => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:37.527485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.659516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600533e-aee2-4989-a919-e2bac8c5b257 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.705020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.705997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.706468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.706925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] PATH_INFO: `/v3/projects/a624bee162274d80bf52fd9ee90c7807` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.708345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] RBAC: Authorizing `identity:get_project(project_id=a624bee162274d80bf52fd9ee90c7807)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.709991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e454aea-b7cf-498f-8348-9d66a070514b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.712233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 213/414] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:37 2022] GET /identity/v3/projects/a624bee162274d80bf52fd9ee90c7807 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:37.717628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:37.827710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435186a-5ee7-4787-981b-0508f296f266 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:37.867875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:37.868978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:37.869454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:37.869920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] PATH_INFO: `/v3/projects/a624bee162274d80bf52fd9ee90c7807/groups/817b80951c0e43509350a88eac4496f0/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:37.874649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a624bee162274d80bf52fd9ee90c7807, group_id=817b80951c0e43509350a88eac4496f0, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:37.876455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246234cf-922c-4a84-88d9-1756f4404de4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:37.895588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:37.900470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 202/415] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 31 03:53:37 2022] PUT /identity/v3/projects/a624bee162274d80bf52fd9ee90c7807/groups/817b80951c0e43509350a88eac4496f0/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:38.256456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb323-6819-4647-8c00-146dcb31c73a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:38.256859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb323-6819-4647-8c00-146dcb31c73a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:38.257056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb323-6819-4647-8c00-146dcb31c73a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:38.258530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 214/416] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:38.263912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca64b05-e5ff-4b6d-8594-49124ab0da58 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:38.264197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca64b05-e5ff-4b6d-8594-49124ab0da58 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:38.264473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca64b05-e5ff-4b6d-8594-49124ab0da58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:38.284934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4ca64b05-e5ff-4b6d-8594-49124ab0da58 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:38.286168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4ca64b05-e5ff-4b6d-8594-49124ab0da58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:38.359220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 203/417] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:38.804670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eb323-6819-4647-8c00-146dcb31c73a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:38.866648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:38.931714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eb323-6819-4647-8c00-146dcb31c73a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:38.968364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551f14-dbc2-4e80-8a17-3de85d981e08 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:38.968941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551f14-dbc2-4e80-8a17-3de85d981e08 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:38.968941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551f14-dbc2-4e80-8a17-3de85d981e08 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:38.975058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 204/418] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:38.980784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b31dd-1243-46ea-8c2d-fce43e64ed40 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:38.981061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b31dd-1243-46ea-8c2d-fce43e64ed40 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:38.981421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b31dd-1243-46ea-8c2d-fce43e64ed40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:38.981931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:38.982667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:38.983051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:38.983433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.011150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:39.017591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c95b31dd-1243-46ea-8c2d-fce43e64ed40 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:39.018788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c95b31dd-1243-46ea-8c2d-fce43e64ed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:39.020838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:39.021781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:39.022616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:39.026680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 215/419] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:39.028785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.138419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 205/420] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:39.139704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739011-5932-4a2c-85e3-af201a84d209 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.140169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739011-5932-4a2c-85e3-af201a84d209 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.140503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739011-5932-4a2c-85e3-af201a84d209 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.141760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 206/421] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:39.146937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.147240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.147509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.165635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:39.166868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:39.193167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d76f-c4eb-45ee-963a-42aae482ccd6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.219661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.234930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.248659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 207/422] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:39.292618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.293244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.293524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.293793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.294466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:39.294767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:39.296211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:39.313489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 216/423] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:39.318984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.473143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4db6-fe69-49a6-a1c0-6fb97fe46aac None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.481156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.527141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.527810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.528085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.528345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.601055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc1601-1d5d-4643-8c6b-5af3d3fa2522 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.602841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.607556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:39.609625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:39.644961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.645628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.645908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.646181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.659833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:39.666910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:39.667827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:39.668711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:39.671167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 217/424] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:39.671984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.706096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.732752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 208/425] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:39.734275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.777839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114b703-f76a-41e6-91ba-0fc95d89b930 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.818849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.819704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.819872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.820021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.832248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:39.837588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:39.839729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 03:53:39.841666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:39.843429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 218/426] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:39.843429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.852837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.865058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bffff-4073-4283-b57f-bcf24479cccb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.902116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.902712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.903093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.903294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.903935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:39.904229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:39.905647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:39.913803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 209/427] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:39.914590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:39.947628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:39.958441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10647a3-3f08-434f-bcc5-53654e58635b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:39.992327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:39.992874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:39.993154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:39.993416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:39.994552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:39.996045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d464e4-1d3e-47c4-be94-45ecb2ee0950 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 219/428] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dc7ea-9491-452d-8e64-5a8325833125 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dc7ea-9491-452d-8e64-5a8325833125 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dc7ea-9491-452d-8e64-5a8325833125 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 220/429] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:40.002988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dc7ea-9491-452d-8e64-5a8325833125 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.033526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f35639-1493-4358-b31e-b22e43f040d3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.068824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.069434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.069708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.069976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.070631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:40.070930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.072357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.085584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 210/430] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.085584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.100323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.113963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dc7ea-9491-452d-8e64-5a8325833125 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.149720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.150623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.151130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.151397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.179955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.190432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21d72-5091-4d4e-a7f7-951608c5cbee None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.225202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.226917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.228943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.230158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.230158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.230158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.231280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.232715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b16ca5-5e0c-43a3-9754-1af2216a4801 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.234604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 211/431] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.235658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.235944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.236217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.255292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:40.256405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:40.335504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 212/432] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:40.336466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.350688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 221/433] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:40.351884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.361974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.421222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.452321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af493e9-8e31-42d2-a99c-0daed4dded32 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.465084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac286933-11f7-4b9e-a04c-c480cc23d61c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.487341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.488098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.488526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.488806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.501597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.502269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.502478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.502748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.503512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=ee67389746c245038655ca9ddc941126, scope.project.id=f2239116cdae485a95fef3677e13e435) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:40.503828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.505369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.515094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 222/434] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=ee67389746c245038655ca9ddc941126 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.515094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.584406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.585900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.603399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.649545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bac651-96e5-4d05-b4a3-0d50156c0ab3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.674514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.687304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.688112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.688492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.688744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] PATH_INFO: `/v3/projects/f2239116cdae485a95fef3677e13e435` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.690044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] RBAC: Authorizing `identity:get_project(project_id=f2239116cdae485a95fef3677e13e435)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.691633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435a6c5-096e-4744-b4fd-2d271ef03bac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.693304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 223/435] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/projects/f2239116cdae485a95fef3677e13e435 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.694434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c947b-6066-46a1-80e7-d786c8d81ddf None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.694710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c947b-6066-46a1-80e7-d786c8d81ddf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.694974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c947b-6066-46a1-80e7-d786c8d81ddf None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.697434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 224/436] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:40.698197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c947b-6066-46a1-80e7-d786c8d81ddf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.705288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 213/437] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:40.706406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.801552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c947b-6066-46a1-80e7-d786c8d81ddf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.806379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563e3b6-b406-4293-824f-9909185eabbf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.819560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.836413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.837029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.837310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.837587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.840526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.841153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.841425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.841692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.842804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.844588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d47ea-643d-4146-aafa-51edd8f7ef35 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.846413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 214/438] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:40.847310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.847589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.847872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.847963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:40.853258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:40.854117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:40.854932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 225/439] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:40.856181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.866165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:40.867390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:40.908220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:40.934070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 215/440] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:40.934883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:40.958294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5133-7333-4aaa-9417-5c3a019f7112 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:40.992680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:40.993407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:40.993684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:40.994072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:40.995218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:40.997200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:40.999231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 226/441] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.000000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.038394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534bf64-4f0c-460b-a821-a66015b10952 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.049769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.074910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.075458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.075732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.076009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.076729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=f2239116cdae485a95fef3677e13e435) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:41.077143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.079001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.095076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 216/442] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=f2239116cdae485a95fef3677e13e435&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.095076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.123458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2590f-60ea-44bc-8af2-219b34e97c5a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.146280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.168849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.169563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.169958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.170322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.171230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:41.171741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.173241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.182799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 227/443] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.188373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.225051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32a472-0135-48da-8bce-0c8494acac48 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.262374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.263001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.263298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.263564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.264662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.266401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.268265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 217/444] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.268798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.314881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b1817-f9a7-4c08-bebe-06d0c2e3231e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.319431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.356709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.356894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.357146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.357419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.393316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c41537-bf10-4fe4-986a-6fe8b869fa7e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.410416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.435260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.435826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.436465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.436777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.436893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.437050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.454129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:41.459718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:41.460629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:41.461422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 218/445] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:41.462138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.563920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 228/446] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 03:53:41.564882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.588460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045ee9-bca2-40f5-8d24-869e14b30503 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.592666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.633540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.634066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.634338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.634606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.635360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=27e940be03fb44c8974bdff2bc53de9b, scope.project.id=46caf8d466c343af892d0910efcfe30e) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:41.635660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.637118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.642932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 219/447] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=27e940be03fb44c8974bdff2bc53de9b => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.643782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.663621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.684272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4d069-d126-4056-9398-2a11eef6384f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.724698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.725321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.725601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.725872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.726547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:41.726848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.728290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.742709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 229/448] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.743931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.773745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2a58e-a2a5-4ff8-a36f-5e21c0c99aa3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.813028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.813662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.813942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.814214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] PATH_INFO: `/v3/projects/46caf8d466c343af892d0910efcfe30e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.815375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] RBAC: Authorizing `identity:get_project(project_id=46caf8d466c343af892d0910efcfe30e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.816819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.818908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 220/449] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/projects/46caf8d466c343af892d0910efcfe30e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:41.824173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.865831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.872290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dec66d-efa7-4d6d-8844-20743da490bd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.913797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.914331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.914615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.915304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.930951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:41.938460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02da7-0313-46f7-86f1-1a9e7bb53224 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:41.982888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:41.984000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:41.984000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:41.984188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] PATH_INFO: `/v3/groups/f950ea0bee09456eb6416d40f58f2ab2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:41.985387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] RBAC: Authorizing `identity:get_group(group_id=f950ea0bee09456eb6416d40f58f2ab2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.986861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:41.989068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 221/450] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/groups/f950ea0bee09456eb6416d40f58f2ab2 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:41.993514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:41.994836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:41.995011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:42.091194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 230/451] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:42.093464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451e5c-7a6e-46a8-b4f6-29d2b4fcb8c4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:42.095013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:42.097305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:42.134041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:42.134663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:42.134939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:42.135222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:42.135949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] RBAC: Adding query filter params (group.id=f950ea0bee09456eb6416d40f58f2ab2, role.id=ee67389746c245038655ca9ddc941126, scope.project.id=46caf8d466c343af892d0910efcfe30e) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:42.136252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:42.137737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:42.144095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 222/452] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:41 2022] GET /identity/v3/role_assignments?group.id=f950ea0bee09456eb6416d40f58f2ab2&scope.project.id=46caf8d466c343af892d0910efcfe30e&role.id=ee67389746c245038655ca9ddc941126 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:42.144631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:42.219493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d10ad-8978-4b2c-bd12-adaa116b6969 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:42.262787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:42.263464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:42.263696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:42.263995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] PATH_INFO: `/v3/projects/a624bee162274d80bf52fd9ee90c7807` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:42.267916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=a624bee162274d80bf52fd9ee90c7807)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:42.269400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:42.271162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 231/453] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:53:42 2022] GET /identity/v3/projects/a624bee162274d80bf52fd9ee90c7807 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:42.277017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:42.382487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdf194-a914-43ee-8b6a-870739ed5282 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:42.416909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:42.417533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:42.417804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:42.418111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] PATH_INFO: `/v3/groups/817b80951c0e43509350a88eac4496f0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:42.419268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] RBAC: Authorizing `identity:get_group(group_id=817b80951c0e43509350a88eac4496f0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:42.420708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa89604-4744-4700-bc0c-e065f25941a8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:42.422486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 223/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 31 03:53:42 2022] GET /identity/v3/groups/817b80951c0e43509350a88eac4496f0 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:42.428887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:42.528010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61242c-453d-439f-bacd-0cae6ee17ff7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:42.541376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:42.567110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:42.567716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:42.567993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:42.568260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:42.568981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] RBAC: Adding query filter params (group.id=817b80951c0e43509350a88eac4496f0, role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=a624bee162274d80bf52fd9ee90c7807) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:42.569285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:42.570740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1008b1-e784-4504-8273-0a1d2872e497 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:42.576602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 232/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 31 03:53:42 2022] GET /identity/v3/role_assignments?group.id=817b80951c0e43509350a88eac4496f0&scope.project.id=a624bee162274d80bf52fd9ee90c7807&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:44.961057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07cca-daba-41c0-9e6d-c01aade1594f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:44.961692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07cca-daba-41c0-9e6d-c01aade1594f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:44.961692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07cca-daba-41c0-9e6d-c01aade1594f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:44.962937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 224/456] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:44.968663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5b75-904a-4d90-90f6-9635b4f59292 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:44.969506 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5b75-904a-4d90-90f6-9635b4f59292 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:44.970005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e5b75-904a-4d90-90f6-9635b4f59292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.000128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7b2e5b75-904a-4d90-90f6-9635b4f59292 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:45.001672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b2e5b75-904a-4d90-90f6-9635b4f59292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:45.088543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 233/457] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:45.122555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567e2b-fb40-4a40-a0c3-0aa4ea2b27e5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.122883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567e2b-fb40-4a40-a0c3-0aa4ea2b27e5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.123125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567e2b-fb40-4a40-a0c3-0aa4ea2b27e5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.124789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 225/458] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:45.129941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddcc-3c83-4659-a971-0a6202578e23 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.130224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddcc-3c83-4659-a971-0a6202578e23 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.130497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddcc-3c83-4659-a971-0a6202578e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.156698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9e85ddcc-3c83-4659-a971-0a6202578e23 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:45.157941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e85ddcc-3c83-4659-a971-0a6202578e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:45.241533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 234/459] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:45.399143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff08af-042f-4c32-a9eb-db88f73a48f8 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.399427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff08af-042f-4c32-a9eb-db88f73a48f8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.399947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff08af-042f-4c32-a9eb-db88f73a48f8 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.402072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 226/460] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:45.407486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.407762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.408215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.436037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:45.437594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:45.462697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff08af-042f-4c32-a9eb-db88f73a48f8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:45.473678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:45.515939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 235/461] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:45.531801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:45.600465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff08af-042f-4c32-a9eb-db88f73a48f8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:45.647154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:45.647762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:45.648393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.648666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.648932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.670740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:45.672302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:45.701302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:45.702701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:45.703778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:45.703778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:45.703778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:45.707153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 227/462] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:45.712020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:45.760136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbb48d-5d74-41f8-bc25-bd240296e8ef None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:45.794991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:45.796677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.797290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.797867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.810663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:45.813263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:45.815369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6df5f1c9-6c3a-4894-a712-ff868edb47e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:45.832777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:45.833711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:45.834547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:45.834547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:45.834547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:45.841027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f85979-e74d-4be7-9378-3041b1cb8b26 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:45.841241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 236/463] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:45.842661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0f9-fe85-4e45-ba30-20dcec0c1ace None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.843226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0f9-fe85-4e45-ba30-20dcec0c1ace None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.843687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0f9-fe85-4e45-ba30-20dcec0c1ace None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.845673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 237/464] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:45.846658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e0f9-fe85-4e45-ba30-20dcec0c1ace None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:45.866382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:45.879583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:45.880453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.880871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.881287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:45.882072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:45.882515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:45.884039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:45.896484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 228/465] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:45.897518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:45.960432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e0f9-fe85-4e45-ba30-20dcec0c1ace None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:45.995183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:45.995954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:45.996550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:45.996820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.004101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377fb56-fdb0-46ca-90fe-927477f0df22 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.012083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.013426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6d7254b7-4fd8-4320-9f4c-8477f1711ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:46.028100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:46.029010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:46.029847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:46.029847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:46.029847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:46.029847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 238/466] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:46.030631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.030795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.031066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.045298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.045845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.046196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.046464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.047137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:46.047442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.048838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.049855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:46.051226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:46.061269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 229/467] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:46.062322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.073512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.121912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.127381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 239/468] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:46.128433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.180850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964425ab-e688-45e0-ade7-cfdc0e65d6df None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.216638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.217367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.217837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.218508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.219051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.220625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.230658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccc32-e5d7-460b-b831-0a998c5c83c4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.245797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 230/469] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:46.247176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.271623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.272580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.273158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.273589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.274478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:46.274948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.276543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.290764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 240/470] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:46.296983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.363221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.380585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d1cdf-3f0d-45a1-af80-42e6e0beaf4f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.391414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.409969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edf667-4f41-4200-9bd7-c25f4c1315c0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.420339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.421257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.421756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.422292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.423156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.424873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bc18f-e5ab-4352-910d-8a7ec4d72267 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.452971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.453826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.454109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.454377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.455897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.457042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 231/471] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:46.462746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963cb2e-848f-4caf-98b4-bf2982f70f12 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.462831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb984e-3afd-42e0-a487-888cfecd9f0d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.462831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb984e-3afd-42e0-a487-888cfecd9f0d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.462831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb984e-3afd-42e0-a487-888cfecd9f0d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.462831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 232/472] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:46.464016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb984e-3afd-42e0-a487-888cfecd9f0d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.484727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 241/473] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:46.486043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.486330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.486601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.520685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:46.521948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:46.625352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 242/474] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:46.632069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb984e-3afd-42e0-a487-888cfecd9f0d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.656813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.676024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.676654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.676931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.677200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.690729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.692251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:46.708611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:46.709563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:46.710436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:46.710436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:46.710436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:46.710436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 233/475] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:46.715846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.769218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:46.827426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31890203-e9bf-4c4d-8533-4c4e68198054 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:46.866179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:46.866779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:46.867075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:46.867348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:46.867984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:46.868297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:46.869659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:46.883500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 243/476] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:46.889296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:46.933694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:47.035491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055cd60-8a0f-4ca2-a3c8-7ff99563d4f6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:47.069627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:47.070233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:47.070510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:47.070777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:47.071491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:47.072914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:47.087053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225189-ca17-4eaf-b528-4db0152ba033 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:47.103410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 234/477] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:47.113145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:47.135302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:47.135929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:47.136272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:47.136608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:47.137349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:47.138801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:47.148508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 03:53:47.148508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 03:53:47.148508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [parameters: {'id': '11a9899662a240399847551119bfd39b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 03:53:47.148508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:53:47.166801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:53:47.167962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 03:53:47.169114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 03:53:47.170129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:53:47.171752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:53:47.172955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'id': '11a9899662a240399847551119bfd39b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:47.174013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 03:53:47.179998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:47.182004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 244/478] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 31 03:53:46 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:47.182141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:47.230972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:47.298898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68b5c-a80d-4842-baff-6f4e5ba8faa8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:47.343834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:47.344478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:47.344800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:47.345071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:47.355762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-55a183ce-087a-4ff3-a532-1d704522ee1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:47.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:47.362109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:47.362904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 235/479] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:47.369834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:47.476040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ea06-bdea-49a3-a135-8e488a27879e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:47.511208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:47.512066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:47.512480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:47.512885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:47.513677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:47.514126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:47.515818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f44b93-8819-4c98-bbbe-cd7160694ad8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:47.524621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 245/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:48.676373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa18096-fac9-449f-b138-9f2229f67725 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:48.677216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa18096-fac9-449f-b138-9f2229f67725 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:48.677678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa18096-fac9-449f-b138-9f2229f67725 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:48.679545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 236/481] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:48.685037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ff5b-adda-45c4-86a4-db5d86ae38d5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:48.685733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ff5b-adda-45c4-86a4-db5d86ae38d5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:48.685733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216ff5b-adda-45c4-86a4-db5d86ae38d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:48.710565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a216ff5b-adda-45c4-86a4-db5d86ae38d5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:48.711800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a216ff5b-adda-45c4-86a4-db5d86ae38d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:48.744630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:48.801762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 246/482] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:49.009918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea26-d876-474d-9c30-015660e2e081 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.010328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea26-d876-474d-9c30-015660e2e081 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.010500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea26-d876-474d-9c30-015660e2e081 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.012269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 237/483] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:49.018230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599d02-e0cc-41d9-bf47-ba434952a135 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.018521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599d02-e0cc-41d9-bf47-ba434952a135 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.018790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c599d02-e0cc-41d9-bf47-ba434952a135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.049374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8c599d02-e0cc-41d9-bf47-ba434952a135 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:49.050565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c599d02-e0cc-41d9-bf47-ba434952a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:49.164446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 247/484] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:49.171004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ea26-d876-474d-9c30-015660e2e081 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.270840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65446ed-1028-438d-a973-384430acf69d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.271166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65446ed-1028-438d-a973-384430acf69d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.271166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65446ed-1028-438d-a973-384430acf69d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.290774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 248/485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:49 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:49.295076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.295355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.295628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.336199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:49.336859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:49.338468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:49.372881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ea26-d876-474d-9c30-015660e2e081 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:49.448399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573d77-b174-407d-9006-051a49594940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:49.449155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573d77-b174-407d-9006-051a49594940 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.449918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573d77-b174-407d-9006-051a49594940 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.450427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573d77-b174-407d-9006-051a49594940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.473947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19573d77-b174-407d-9006-051a49594940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:49.483372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 249/486] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-19573d77-b174-407d-9006-051a49594940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:49.483546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:49.484386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:49.486205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 238/487] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:49.491139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.514567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573d77-b174-407d-9006-051a49594940 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.527506 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:49.649163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573d77-b174-407d-9006-051a49594940 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:49.714134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08262-9c29-44c4-a07e-9bd50ef34d13 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:49.747799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:49.748508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.748791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.749124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.762061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:49.762679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.762956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.763268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.763924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:49.764224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:49.766018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:49.771726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:49.777022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:49.778155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:49.778958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 239/488] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:49.786127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.797202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 250/489] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:49.798654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.813253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:49.893427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:49.911417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590a25-962c-4d6d-a54d-57aa5dedfddb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:49.942800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f2248-79c9-4411-9f00-ff72dd6f6e61 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:49.947999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:49.948720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.949066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.949409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.966129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:49.968530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:49.984141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:49.984749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:49.984851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:49.985692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:49.992062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:49.992062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:49.992062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:49.992229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:49.992229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:49.992229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:49.992229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:49.992465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 251/490] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:49.992465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:49.992582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc22fd-a8d4-47d6-b4fb-7e8afb7d624c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.003125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 240/491] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:50.003125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e071-7b19-4337-acd6-543fad3be3e4 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.003125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e071-7b19-4337-acd6-543fad3be3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.003125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e071-7b19-4337-acd6-543fad3be3e4 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.003613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 241/492] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:50.004340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18e071-7b19-4337-acd6-543fad3be3e4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.078478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.101771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29a844-59d6-4bb6-946f-e7dfed37756d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.113201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18e071-7b19-4337-acd6-543fad3be3e4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.143562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.144204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.144481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.144747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.147522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.148886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.149676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.150021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.150287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.150928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:50.151242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.152632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3bf47-b242-4b01-a3aa-49be2fdf83d3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.156953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:50.162183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:50.163078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:50.163859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 252/493] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:50.164516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.164591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 242/494] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:50.169990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916a1c9-9158-4744-b090-0076f6a3bb94 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.169990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916a1c9-9158-4744-b090-0076f6a3bb94 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.169990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916a1c9-9158-4744-b090-0076f6a3bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.184715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f916a1c9-9158-4744-b090-0076f6a3bb94 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:50.185908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f916a1c9-9158-4744-b090-0076f6a3bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:50.252137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 243/495] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:50.253546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ece3f-0f20-40d8-bc9e-4a6d66b78524 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.253832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ece3f-0f20-40d8-bc9e-4a6d66b78524 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.254104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ece3f-0f20-40d8-bc9e-4a6d66b78524 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.255369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 244/496] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:50.256066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ece3f-0f20-40d8-bc9e-4a6d66b78524 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.277154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab9263-fd5b-4288-9fdc-a4a0cabcb3e8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.316994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.317637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.317921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.318172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.330589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.330775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.332013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:50.347105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:50.348009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:50.348844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:50.348844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:50.348844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:50.348844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 253/497] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:50.349071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.382655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ece3f-0f20-40d8-bc9e-4a6d66b78524 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.401284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.420777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.421580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.421855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.422121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.429787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.431273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d642d124-d791-440d-9c64-c24b2dc33a57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:50.444030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:50.444918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:50.445961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 245/498] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:50.446759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.446759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.446876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.457300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb69b-fadf-42a5-a710-2e439f2a7591 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.465019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:50.466226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:50.494160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.494768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.495076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.495343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.496044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:50.496616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.497988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.506587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 254/499] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:50.508131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.534515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 246/500] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:50.543990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.594004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.617007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3197d99-9f88-414d-9674-83067500fb29 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.628128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.653024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.653631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.653908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.654181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.654822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:50.655343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.656840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.659822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4ab6c-af06-487d-8c33-6966caee0031 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.668401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 255/501] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:50.669179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.709415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.710420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.710702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.710976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.711746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:50.712398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.714165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.751115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 247/502] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:50.751952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.777937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f0eb-f971-4132-add6-96aa99ce6149 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.814374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.815006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.815312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.815582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.825296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.826742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:50.841313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:50.842170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:50.843258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:50.843258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:50.843258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:50.843258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 256/503] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:50.843570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.853791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9af484-5523-4941-b940-1819a030aad5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.861297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.869173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:50.888831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.889455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.889798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.890226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.900532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:50.905393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:50.906456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:50.907290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 248/504] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:50.907938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:50.948268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440912d-c598-4a90-95b6-dc743833cd56 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:50.982134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:50.982759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:50.983051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:50.983412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:50.990879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:50.992353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:51.005049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:51.005954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:51.006828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 257/505] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:51.007405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.007470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab81c-5b36-4526-ad02-2165fef3b73c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.041705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.042299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.042573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.042827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.076565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.088564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.107857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b141-d49f-409b-bcc0-5638c2849843 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.111421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.112919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.144009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.144628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.144902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.145164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.155947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.157456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:51.171882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:51.172806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:51.173686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:51.173686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.173686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:51.173686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 258/506] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:51.173907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.214582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 249/507] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:51.215458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.273238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1858a-9718-4cc9-9755-4ae34d5e972f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.282835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.310643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.311281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.311675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.312000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.312629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:51.312923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.314322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.315759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.318635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb14b38-5691-4ee4-a341-5bcdf94bd95d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.326685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 259/508] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:51.327920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.354902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.355723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.356116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.356396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.357060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:51.357364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.358908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.366780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 250/509] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:51.367601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.430419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f0f5e-2e92-422c-9778-67a4f1376a79 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.465861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.466038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66496109-7aba-4cc6-ac29-368d4badc2ec None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.466464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.466743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.467014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.467812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:51.468293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.469748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.493823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.502114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.502715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.502989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.503286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.503913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:51.504214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.505607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.506060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 260/510] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:51.506875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.517985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 251/511] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:51.518903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.562816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.609036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1519-51b0-4370-b319-e4eb720d2389 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.618489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a24e-4309-43ca-9fde-8dfe3b187e99 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.642266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.642879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.643189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.643461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.652329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.652951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.653227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.653420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.653524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.654910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.660419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.661862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:51.669648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:51.670810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:51.672111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:51.672111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.672111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:51.672111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 261/512] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:51.672111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:51.674657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:51.675558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:51.676423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 252/513] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:51.676976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.727921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.778688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fc75f-cf43-46e1-98c0-8ff72be545cc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.781663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855807d9-ec6e-42b9-bdae-202eaee1a667 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.783325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.812796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.813404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.813690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.813980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.814307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.814927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.815228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.815493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.825274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.826816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:51.841054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:51.842080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:51.843094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:51.843094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:51.843094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:51.843094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 262/514] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:51.843094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:51.884606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.886014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.932383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:51.946282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9a1b1-ffcc-4f1c-b217-79c983d88238 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:51.982374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:51.982981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:51.983371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:51.983798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:51.984454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:51.985031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 253/515] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:51.985943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:51.986383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.008473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 263/516] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:53:51 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:53:52.009284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.026763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.109953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617e0a-ef18-4530-8620-f44a4f4e21b9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.130840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e7c53-da6a-4ecb-be9b-6e137735b168 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.150232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.150794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.151062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.151350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.151977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:52.152271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.153639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.166961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 254/517] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:52.167804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.176533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.177141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.177415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.177692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.178612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:52.179535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.180302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.180429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.204173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 264/518] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:52.204940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.276567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.285877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51571bf5-c3f2-4b33-bc4b-22ab4c9480c1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.311902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71625e-119f-4a3c-8953-7c798baf157c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.322412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.323046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.323323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.323590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.324583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:52.324891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.326611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.339249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 255/519] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:52.340090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.357881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.358621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.358894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.359431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.371532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.372954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:52.388701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:52.390031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:52.391755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:52.391755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:52.391755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:52.394520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 265/520] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:52.395276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.417177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.455657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949406b0-a443-4d9a-bc87-87e1f235aa95 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.492997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.493778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.494124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.494455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.499746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32070dba-e67f-4608-b515-8d3a9aa9e399 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.502028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.503595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:52.515858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:52.516932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:52.517975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:52.518404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 256/521] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:52.519147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.529259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.542335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.542973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.543319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.543618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.614225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.615808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.621025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fee467-1d7d-4470-8f74-9f232f34e02b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.649036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.659200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.660183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.660462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.660731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.671790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.673212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:52.688353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:52.689336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:53:52.690385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:52.691141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 257/522] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:52.695936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.728961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 266/523] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:52.728961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.748490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.809821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef87c3-a192-4e9f-ac31-cab4db040189 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.824371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8ff11-7d33-42ca-8d84-0ded50a48bdd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.844917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.845605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.845897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.846164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.846800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:52.847102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.848700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.858474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:52.859220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:52.859497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:52.859765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:52.860445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:52.860748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:52.865495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 258/524] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:52.865638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:52.865729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.889015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 267/525] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:52.889015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:52.918637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.971579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:52.971771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199c75-c689-4476-be1d-9af0672ccdab None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:52.991791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc86f7-7e34-4bab-93fc-4449025add2d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.006347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.006951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.007255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.007519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.008247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:53.008542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.009972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.026647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.027307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.027583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.027847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.032717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 259/526] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:53.032717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.043234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.044730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:53.059979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:53.061068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:53.062106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:53.062106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:53.062106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:53.062106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 268/527] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:53.062487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.140409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397c5b4-d4b0-4a56-a0f1-9bb5f50b70a0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.150353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.174399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a8c3c-20a4-42a9-88d0-eb2492daa93e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.183767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.184400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.184977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.185254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.192302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.193789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:53.205332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:53.206425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:53.207541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 260/528] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:53.208007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.215230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.215459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.215738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.216005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.216731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:53.217038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.218545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.220523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.226560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 269/529] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:53.228343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.330756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b5ed8-1a89-4826-af68-bc7613c100e5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.369443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.371498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.371998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.372482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.374209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055537e-27c6-4a6d-b842-010a452c2b79 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.411296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.423174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.424408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.424833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.425526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.426346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:53.426811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.429284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.446889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 270/530] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:53.448232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.451473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.453282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.529609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.565869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 261/531] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:53.565869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.593484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c46b5-feca-4587-8af8-6a50229aca65 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.632388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.641766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.642867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.643742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.644163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.645036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:53.645495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.647789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.662221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 271/532] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:53.663200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.677227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80626f-6c68-4baf-a91f-84f7e292000d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.723710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.724907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.725523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.726038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.735652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.737607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:53.753486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:53.754567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:53.755631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:53.758088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 262/533] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:53.759006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.776575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573cc1-3bdb-475a-9074-d2fbb87b8833 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.803546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.815690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.816438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.816813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.817256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.829586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.831263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:53.846957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:53.848111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:53.849158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:53.849158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:53.849158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:53.854666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 272/534] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:53.854666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.877036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cb1da-28ca-448b-a990-fdad8c7c8086 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:53.903965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:53.913883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:53.914673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:53.915151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:53.915566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:53.916435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=a77972e7e64f47c7a3fe7153841976c3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:53.916884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:53.918490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:53.931332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 263/535] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/role_assignments?user.id=a77972e7e64f47c7a3fe7153841976c3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:53.932567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:53.965783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9785d36-6054-45c5-95a1-9d596bc521e7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.006094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.006910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.007291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.007669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.008519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:54.008917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.010599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36938b4-eba7-49b1-8f87-d7a0450f7d89 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.030463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 273/536] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:54.032335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dff30-a033-4d20-8b19-0941af32fd88 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.032722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dff30-a033-4d20-8b19-0941af32fd88 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.033087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dff30-a033-4d20-8b19-0941af32fd88 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.035197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 274/537] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:54.036284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dff30-a033-4d20-8b19-0941af32fd88 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:54.050378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067dc71-9f62-4faf-a251-fe868b4ba643 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.076754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:54.090240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.090861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.091176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.091454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.092103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:54.092402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.093836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee720-42af-46d1-a17b-4aeb0e9e9253 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.106286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 264/538] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:54.107681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.107966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.108346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.127140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:54.128398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:54.171409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dff30-a033-4d20-8b19-0941af32fd88 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.182890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:54.215179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 265/539] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:54.216364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:54.228692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.229643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.230078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.230490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.231465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=9a1ecf64ce544ee5aa7839acde57bea2) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:54.231942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.234938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.246674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 275/540] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/role_assignments?user.id=9a1ecf64ce544ee5aa7839acde57bea2&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:54.360109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcadee6-24ff-4ae7-96d1-e7739b551372 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.401033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.401905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.402188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.402460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.410732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.412269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:54.425696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:54.426768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:54.427867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 266/541] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:54.436166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:54.559546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:54.570303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36308269-23fa-45ad-a0b9-897cb5ea4e99 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.615833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.616458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.616808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.617148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.617916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:54.618290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.619931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.634941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 276/542] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:54.644174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:54.707688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:54.712513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:54.833303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7674606-4b40-45cf-901a-9bf78d697863 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.867695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec2544-53e6-4d09-a0ac-8148ab1782b9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:54.908859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.909496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.909749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.910021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.910761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=8d87047620874e09bc5d851978949acc) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:54.911090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:54.914113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:54.925986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:54.926618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:54.926897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:54.929715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:54.935509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 267/543] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/role_assignments?user.id=8d87047620874e09bc5d851978949acc&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:54.949658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:54.959182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:54.960115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:54.961620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:54.963049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 277/544] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:54.969122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:55.182009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853ddbd-1b8d-4288-9de5-ff2339bbc7a2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:55.188042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:55.257796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:55.258761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.259259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.259718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.260592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:55.261079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:55.262651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73d51a-21d7-4b3b-acfd-69e3e6215670 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:55.274152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 268/545] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:55.280744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:55.302267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:55.447541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e60e23-b692-4881-8255-50d69cadb0d4 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.447943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e60e23-b692-4881-8255-50d69cadb0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.448319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e60e23-b692-4881-8255-50d69cadb0d4 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.449747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f5084-8bfe-43b6-9474-9080c08933b2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:55.450272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 269/546] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:55.460453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.460735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.461004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.491475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:55.492112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.492390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.492664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.499426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:55.500649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:55.516988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:55.518475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:55.542411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:55.543408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:55.544246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:55.544246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:55.544246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:55.544246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 278/547] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:55.550658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:55.599514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 270/548] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:55.666488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:55.707855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9258388-f2e3-4f07-ad89-a684663014e6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:55.754293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:55.754899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.755233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.755511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.756147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:55.756440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:55.757804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:55.774323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 279/549] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:55.779868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:55.885794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:55.892752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad0c0b-482f-4cd7-950f-e6d40740b562 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:55.937735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:55.938490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:55.938645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:55.938878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:55.946227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:55.948520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:55.955829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:55.961824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:55.963302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:53:55.964379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:55.966843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 271/550] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:53:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:55.972811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.119526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad2ec1-8705-40d6-b923-119e70693542 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.155244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710de66-da73-4cb2-9737-7b6f6e16af08 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.169405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.170017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.170297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.170565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.172241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:56.172553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:56.174571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca899e-4741-4376-b85b-17c2c5efe912 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:56.193216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.196683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 272/551] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:53:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:53:56.203082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec5b5-c273-4535-b1ff-fadee46b3416 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.203082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec5b5-c273-4535-b1ff-fadee46b3416 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.203082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec5b5-c273-4535-b1ff-fadee46b3416 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.206075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 273/552] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:56.206939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec5b5-c273-4535-b1ff-fadee46b3416 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.211165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.218055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.218753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.219070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.229976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:56.230245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e5cb60fb-7ba3-4876-bbb0-e50b7dbb5191 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:56.235827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:56.236917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:56.237736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 280/553] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:56.238455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.243693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.243693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.259440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:56.260640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:56.339161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 281/554] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:56.340278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.346969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec5b5-c273-4535-b1ff-fadee46b3416 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.384517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.385505 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.386004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.386474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.459616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:56.461473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:56.481621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.598407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 274/555] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:53:56.604190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.625202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92d85-b7a0-4f9d-a895-64194bed6371 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.667381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.695455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.696253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.696530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.696794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.697451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:56.697750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:56.699245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:56.708282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 282/556] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:56.708518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e644-042e-4cd5-bf68-da50fd2095de None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.709165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.732510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.754798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.755595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.755869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.756135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.766395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:56.768347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-997fedf8-0f21-45db-a235-e1c6890b77fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:56.783581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:56.784544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:56.785975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:56.785975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:56.785975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:56.785975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 275/557] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:53:56.786203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab80bbb-c4a7-46f8-a5ec-f4de3e6cae87 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.786416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab80bbb-c4a7-46f8-a5ec-f4de3e6cae87 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.786673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab80bbb-c4a7-46f8-a5ec-f4de3e6cae87 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.788104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 276/558] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:56.792466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab80bbb-c4a7-46f8-a5ec-f4de3e6cae87 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.816111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1878d48-a956-4a22-ba32-e77d452a9e5b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.848501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.849240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.849442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.849707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.862015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:56.870526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:56.872436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 283/559] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:56.875685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:56.918017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab80bbb-c4a7-46f8-a5ec-f4de3e6cae87 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:56.921738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:56.952301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:56.953413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.953413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.953599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.964562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:56.966408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-de16bb61-bb73-4401-a740-ee997fc1068a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:56.981890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:56.982965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:56.983943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:56.983943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:56.983943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:56.983943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 277/560] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:56.984757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:56.984757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:56.984925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:56.994785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fc580-d6a5-4782-9341-134f3fc02503 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.005589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:57.007146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:57.022753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.031384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.031996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.032257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.032522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.033150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.033441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.034793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.048116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 284/561] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.048525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.084969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 278/562] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:57.085813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.161657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.166419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a32-f0b1-4566-8a22-1002cf036015 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.187814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907553b-507e-463f-abbf-b0dd26b4927c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.207926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.209069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.209586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.210052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.211217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.211974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.213666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.223572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.224101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 285/563] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.225193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.228491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.228775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.229099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.230190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.230190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.231577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.244331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 279/564] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.245160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.256831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.345279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705331-3102-432c-b66b-d61a309f7b2c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.349173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547fa4-b1fc-42cc-bf7e-63137cd47b84 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.381649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.382277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.382552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.382819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.385137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.385766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.386043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.386311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.389795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.391275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.396985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.398594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:57.403009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:57.404064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:57.405091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 286/565] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:57.405572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:57.414566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:57.415539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:57.416405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:57.416405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:57.416405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:57.416405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 280/566] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:57.417277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.430427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.476962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.509827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7be4e-62ce-483c-a6d8-86b15fa615ba None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.519298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2069ab-4f36-4914-9424-9563a8672a5e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.542858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.543557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.543833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.544099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.552092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.553539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.554063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.554689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.554964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.555259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b9768f33-62d5-4500-ab5e-df3034ed115f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:57.565290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:57.566388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:53:57.567375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:57.567908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:57.567979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 287/567] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:57.567979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc168e23-9760-427d-a610-5eb2b05d074b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.567979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc168e23-9760-427d-a610-5eb2b05d074b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.568530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc168e23-9760-427d-a610-5eb2b05d074b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.569583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 288/568] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:53:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:57.571515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:57.572586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:57.573418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 281/569] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:57.574095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.574505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc168e23-9760-427d-a610-5eb2b05d074b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.659474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.673966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdb261-b20d-496f-880f-b53b0de80ae7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.682104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc168e23-9760-427d-a610-5eb2b05d074b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.694336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.708767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.709365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.709659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.709922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.710557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.710852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.712288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.716331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.716928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.717230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.717531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.718213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.718511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.719956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf443f-440a-4445-b8a2-7d0bc1952812 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.723882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 282/570] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.724727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.732150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 289/571] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.733571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.733854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.734123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.751673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:53:57.752871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:53:57.820590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 290/572] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:53:57.821334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.826117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d2fc-1cf1-440d-a64d-4f48b663072c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.860994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.861735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.862105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.862475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.863322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.863715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.865255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.883805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 283/573] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.884701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.902805 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.926701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:57.934421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284b10-fa51-4793-9ccf-99e5c1bbb156 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:57.968025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:57.968710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:57.968987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:57.969252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:57.969911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:57.970204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:57.971605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:57.984454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 291/574] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:57.984454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:57.995094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ecd0f-f302-4e7b-9d14-acdb004d3c60 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.032525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.033180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.033453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.033718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.041060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.042488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:58.054051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.055008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:53:58.055979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:58.059767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 284/575] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.060591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.082772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745b1ad-8a78-497f-bcdc-36d0b5dc3416 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.111372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.117115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.117712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.117987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.118252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.118966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=b526cee1dc234083a7f4897fee8e22b6) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:58.119297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.120704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.126649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 292/576] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:53:57 2022] GET /identity/v3/role_assignments?user.id=b526cee1dc234083a7f4897fee8e22b6&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:58.127436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.170150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.174414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd548-3f53-4c47-9975-91816ece9263 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.213489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.214137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.214414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.214683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.226311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.227831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.241954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f92360-6d90-4c65-a5aa-4b350ddb479f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:58.243276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.244199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:58.245081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:58.245081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.245081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:58.245081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 285/577] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.245737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.292125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.292876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.293214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.293545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.304808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.306360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:58.320492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:58.322454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:58.322454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.322454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:58.322454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 293/578] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.322454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.370192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097dc4-f06f-4450-94b4-e54ed8c48a96 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.383960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.407516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.408225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.408498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.409393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.409663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:58.409974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.411440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.430670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 286/579] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:58.431540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.444340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.445581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e151a3-bf64-443a-ae34-441ecf59cdf4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.488157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.488785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.489085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.489822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.504557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:53:58.510088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.511159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 294/580] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.512488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.541093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7ddb6-45f9-4b27-80da-f357fbec0ffb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.580354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.581374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.581967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.582234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.582889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:58.583217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.584592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.600839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 287/581] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:58.600839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.617784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f0cbc-c595-4194-8351-5b877555d9c5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.627306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.651726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.652328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.652599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.652862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.653491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:58.653782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.655226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.666877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 295/582] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:58.667694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.673293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.710129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7a5c-5dd0-4c0b-9081-9e7bec4dee46 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.747757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.748388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.748688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.748998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.762021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.763526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.775683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3080a-4e64-44da-a6c1-5ec9c5df90cf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:58.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.779267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:58.780265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:58.780265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:58.780265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:58.780265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 288/583] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.780265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.813906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.814516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.814782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.815068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.815763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:58.816012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.817363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.827675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 296/584] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:58.828552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.861414 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.890016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557ebb8-a9a7-40ea-991e-aecda0bc9420 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.910568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:58.928008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.928945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.929228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.929633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.936403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.937143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16778-bc33-4ebe-a1ad-d384ce012938 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:58.937843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:58.949071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:58.949923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:58.950788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 289/585] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:58.951939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:58.978468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:58.979136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:58.979417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:58.979681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:58.987579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:58.988998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:59.000589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:59.001555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 297/586] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:59.002466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.062310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd11088-eddf-455b-bd59-db250bd5b520 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.101429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.101640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.102763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.103006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.103314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.103981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:59.104311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.105637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.114616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fc14e-67eb-4748-b5d5-1f722b5c7e33 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.122718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 290/587] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:59.122718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.157328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.157869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.158142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.158406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.164826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.169338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.171072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:59.185330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:59.186626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:59.187555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:59.187555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:59.187555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:59.189038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 298/588] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:59.193614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.232070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d6d63-d7bb-48f3-8f99-fbe5387d68da None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.271771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.272388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.272664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.272929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.273628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:59.273927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.275395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.294802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 291/589] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:59.300423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.300830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f030d1-d151-469d-a72e-1c18403dd4fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.336945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.338665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.339175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.339642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.340560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:59.341076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.343380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.361252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.370413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 299/590] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:59.372104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.407060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.410362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27d6e-bab7-4dd6-b066-67b76adac806 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.447371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.447999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.448270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.448533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.455457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.456927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:53:59.469312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:59.470212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 292/591] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:59.471141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.485030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea831c5-8b31-44f2-892b-986128c1fba1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.523197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.524034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.524456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.524868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.525767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:59.526236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.527962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.540431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 300/592] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:59.541400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.578182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a6767-97d5-4b1a-a117-d7c08c28a46d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.606313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.614440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.615138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.615422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.615694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.633125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=ed122ea299374f9cb2b8b4854df14d65, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.634718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.649895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 293/593] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:59 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:53:59.656995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e8e64-d2fa-417d-a76a-95ffc8cbd418 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.660494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.680512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.701837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.702555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.702841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.703189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.719629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.721644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:53:59.743358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:59.744375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:53:59.745239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:53:59.745239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:53:59.745239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:59.747915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 301/594] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:59.749623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.798000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa686159-52d0-4411-ab63-c360cb59914d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.837809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.838414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.838697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.838963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.839723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:53:59.840021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.841463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.854734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 294/595] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:53:59.855614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.888041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4d47f-f61e-4a22-ab58-e9db15fdb738 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:53:59.907897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.932677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:53:59.934024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:53:59.934728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:53:59.935105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:53:59.936775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:53:59.946404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:53:59.948235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:53:59.963765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:53:59.964729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:53:59.965664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:53:59.968281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 302/596] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:53:59.969313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:53:59.971994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9c68-a915-460e-a3ac-5b68fe2b13f9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.009019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.009632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.009907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.010169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.010816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:00.011137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.012524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.030611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 295/597] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:00.031916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.074148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00ba71-a941-44d5-aa6f-9fe07c7ba989 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.109038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.109680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.109962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.110238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/a77972e7e64f47c7a3fe7153841976c3/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.125097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=a77972e7e64f47c7a3fe7153841976c3, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.126548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.139618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 303/598] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:53:59 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/a77972e7e64f47c7a3fe7153841976c3/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:00.140425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.141606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edea4b6-3981-4198-b58e-5f53b40bd627 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.178018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.189785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.197553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.198981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.198981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.198981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.199571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:00.199922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.201480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc96bae-87ac-4600-b805-598820987afc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.225134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 296/599] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:00.226634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ea43-2085-45f2-a139-dc5c3033dd79 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.226900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ea43-2085-45f2-a139-dc5c3033dd79 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.227194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2ea43-2085-45f2-a139-dc5c3033dd79 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.232632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 297/600] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:00.232632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ea43-2085-45f2-a139-dc5c3033dd79 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.283760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a382-0a81-4fe6-9312-dd086cbcc5d4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.323173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.324154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.324705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.325034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.331928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.333462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c659820b-69c4-41ed-8669-f85e09b38ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:00.346240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:00.347189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:00.348084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 304/601] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:00.348822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.349096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.349339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.359813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2ea43-2085-45f2-a139-dc5c3033dd79 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.373739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:00.375322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:00.398449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.399310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.399703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.400100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.412492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.414038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:00.430154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:00.431432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:00.432542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:00.432542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:00.432542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:00.433840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 298/602] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:00.434781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.466775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 305/603] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:00.467994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.484550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.493471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.589782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb5934-bf66-4516-ba16-ae3a2a286b0c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.630636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.631535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.631992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.632405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.633196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:00.633576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ccbfc-a12a-440c-a414-2c7df6c2f866 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.633806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.635468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.648310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 306/604] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:00.654115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.708359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498a97-4134-4706-864d-074c0114495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.709329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a97-4134-4706-864d-074c0114495e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.710683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a97-4134-4706-864d-074c0114495e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.712391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a97-4134-4706-864d-074c0114495e None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.713301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498a97-4134-4706-864d-074c0114495e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:00.715092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498a97-4134-4706-864d-074c0114495e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.716757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498a97-4134-4706-864d-074c0114495e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.745253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 299/605] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:00.751286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498a97-4134-4706-864d-074c0114495e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.800761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.840484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c998e-45e2-467e-bcdc-a1e427c76e7b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.878869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.880251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.880734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.881418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.882393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:00.885619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498a97-4134-4706-864d-074c0114495e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:00.893605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.895815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:00.911320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:00.912387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:00.913283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:00.916106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 307/606] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:00.917127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:00.932038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:00.933433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:00.934009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:00.934444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/9a1ecf64ce544ee5aa7839acde57bea2/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:00.955137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=9a1ecf64ce544ee5aa7839acde57bea2, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:00.957684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:00.975139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 300/607] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:00 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/9a1ecf64ce544ee5aa7839acde57bea2/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:54:00.980981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.082140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8c99-9670-4add-a72c-b71a95b2cae5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.122131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591772e-1202-4ea7-aced-b26013f6e8bc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.152112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:01.163460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.164103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.164381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.164648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.176144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.176838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.177027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.177294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.177999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:01.178299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.179810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.188861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:01.195996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:01.196934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:01.197773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:01.200483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 308/608] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:01.204720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 301/609] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:01.205983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.210784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.223597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:01.313993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21147fb8-1bf9-481a-8538-9ebf16c6a69a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.320330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee995066-ea90-4707-bc00-849a655b19d1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.352020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.352546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.352827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.353095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.353754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:01.354060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.355466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.357069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.357724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.358048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.358411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/8d87047620874e09bc5d851978949acc/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.364724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 309/610] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:01.370085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.375307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=8d87047620874e09bc5d851978949acc, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.376814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.392807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 302/611] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:01 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/8d87047620874e09bc5d851978949acc/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:54:01.461038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:01.511687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a77f0-11e9-41f4-aedf-5b63b3f59f99 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.547739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.548385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.548665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.548933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.561547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.563052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:01.578081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:01.580369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:01.581651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:01.581651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:01.581651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:01.581651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 310/612] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:01.588124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.655729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:01.701336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191597f8-5880-44f4-8646-c9e88058b4c1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.738006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.738614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.738887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.739183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.739825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:01.740118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.741485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce8f5e-91a0-40f4-8f04-77b2e99e96ba None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.753770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 303/613] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:01.759446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:01.846643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261518d-cf79-4b4b-a7fa-ce8e7ca1ac23 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.846854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261518d-cf79-4b4b-a7fa-ce8e7ca1ac23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.847124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261518d-cf79-4b4b-a7fa-ce8e7ca1ac23 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.848758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 304/614] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:01.853953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.854232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.854729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.880471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db41f06-fbe2-4f3a-9b07-724b3dd5e6d9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:01.887999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:01.895066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:01.895939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:01.936333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:01.937601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:01.937885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:01.938155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:01.945666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:01.947189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:01.959653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:01.960631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:01.961550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 311/615] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:01.966602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.031260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 305/616] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:54:02.114399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bade8-1acb-4d5a-bdb4-13b050b44dd2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.168064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.168708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.168982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.169258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.169995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:02.170297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:02.172212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b36d81-2d9c-41aa-af1a-1c4924c8d2b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:02.193131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 312/617] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:54:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:02.200017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.238854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd73a68-68bd-43e5-81d8-0ac7ca7646e9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.239213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd73a68-68bd-43e5-81d8-0ac7ca7646e9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.239443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd73a68-68bd-43e5-81d8-0ac7ca7646e9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.241030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 313/618] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:02.245754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.246353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.246623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.265994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:02.267228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:02.278128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.296119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.359264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 314/619] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:54:02.376842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43903620-99fb-483f-9556-89dc6ec26b97 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.411752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.412381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.412659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.412926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.423556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:02.424974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:02.440324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:02.441298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:02.442192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:02.442192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:02.442192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:02.442192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 306/620] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:02.447518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.473542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.558590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f7c0-cc86-4406-bbca-092578e5249f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.593503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.594478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.594960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.595592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.596436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:02.596937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:02.598516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:02.612148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 315/621] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:02.619155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.655378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.703262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24acedb-0206-4c86-b136-371ef48f1700 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.707330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.750798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.751935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.752585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.752958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.771442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:02.778002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190ef7f-4399-46bb-8ed0-077f54ec113d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:02.778441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:02.779593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:02.780744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:02.782581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 307/622] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:02.787303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.821550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.822308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.822592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.822863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.831309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:02.832927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:02.845901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:02.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 316/623] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:02.848432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.899333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5d20-2903-49ad-a2b0-19f9cb63c065 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.927865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.938010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.939165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.939669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.940138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.951646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:02.954569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d19e8-198a-48bd-8462-c00774108788 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:02.957817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:02.958757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:02.959685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:02.962461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 308/624] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:02.963543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:02.990673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:02.992430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:02.993243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:02.993686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:02.994177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:02.994982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:02.995465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:02.997283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.006183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 317/625] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:03.007284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.083576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874e077-ccee-436b-a1c4-3ea7e792d97a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.108605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe422e-9510-4262-aaf6-903709f01a99 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.122729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.123678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.124228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.124729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.125563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:03.125972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.127934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.141957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 309/626] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:03.143308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.145551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.146352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.146799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.147261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.148071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:03.148548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.150094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.152862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.160321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 318/627] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:03.161251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.237263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.260771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5bbd-3d9a-4ecc-92a2-e2cca18e2a17 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.267547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d20c65-5f2e-4481-99f1-3099d9d116f4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.301306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.302136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.302497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.302869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/b526cee1dc234083a7f4897fee8e22b6/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.312122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.312963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.313387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.313845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.318847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=b526cee1dc234083a7f4897fee8e22b6, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.320426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.326569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.328716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.337119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 319/628] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:03 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/b526cee1dc234083a7f4897fee8e22b6/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:03.338260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:03.346183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:03.347437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:03.348575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:03.348575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:03.348575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:03.354738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 310/629] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:03.356393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.407970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.497092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7b2ab-7747-4aff-b7c3-2e2472436952 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.501845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32faf4-1d39-468a-af4d-e96db74124d6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.565773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.566666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.566949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.567268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.567924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:03.568257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.569121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.569631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec8ed-335c-4f68-9ade-34e404f21c57 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.569792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.570062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.570329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.571493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.587658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.587658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.591227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 311/630] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:03.592726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1910-ce2f-4fa7-a77f-3cf18a91608a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.593148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1910-ce2f-4fa7-a77f-3cf18a91608a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.593583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1910-ce2f-4fa7-a77f-3cf18a91608a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.599171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 312/631] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:03.600719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.601774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.602253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:03.609878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:03.611089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:03.612233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:03.612233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:03.612233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:03.613166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 320/632] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:03.614330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.629746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:03.631546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:03.706559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 313/633] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:03.707435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.732566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33792ce-aa4e-445b-97b8-dea464b4e269 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.759389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.777432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.778384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.779106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.779523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.790106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.792944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dc82e2e7-139a-43cc-bca5-fc9e965f2202 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:03.807377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:03.808584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:03.809773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:03.812631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 321/634] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:03.813966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a33b-afe3-48e3-bd7c-b337939ca188 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.814412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a33b-afe3-48e3-bd7c-b337939ca188 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.815421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a33b-afe3-48e3-bd7c-b337939ca188 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.817578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 322/635] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:03.818422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402a33b-afe3-48e3-bd7c-b337939ca188 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:03.839545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444c8ce-a233-41f1-be1c-a7328a594d42 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.883162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.884250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.884667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.885025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.885819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:03.886233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:03.888168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be90f8-cab2-47ce-b034-aea002489d94 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:03.902953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:03.903622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 314/636] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:03.911331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.911635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.911910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.937423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:03.939253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:03.951687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402a33b-afe3-48e3-bd7c-b337939ca188 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:03.995812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:03.996990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:03.997622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:03.997908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:03.999239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:03.999555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.001573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.016885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 315/637] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:04.018063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.031946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 323/638] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:04.031946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.055405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.129122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd735c1-eca2-4096-976b-49f86936b459 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.144818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.160754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67a518-950d-4f96-b16a-46af6df72b9b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.169023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.169727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.170159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.170557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.178424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.179986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:04.192594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:04.193618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 03:54:04.194558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:04.198591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 316/639] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:04.199522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.204910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.205844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.206299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.206762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.220895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:04.227475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:04.228416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:04.229257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:04.231421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 324/640] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:04.236284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.322297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1598-ce7e-49ec-8be0-b940a074b617 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.342178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.359922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.360571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.360854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.361124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.366262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ed13-e2d9-4e7e-834c-e676abf8b0eb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.405447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.410550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e27b93-083e-412b-a83e-114497453464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.411465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e27b93-083e-412b-a83e-114497453464 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.412074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e27b93-083e-412b-a83e-114497453464 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.412932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e27b93-083e-412b-a83e-114497453464 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.413961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e27b93-083e-412b-a83e-114497453464 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:04.414446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e27b93-083e-412b-a83e-114497453464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.416516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e27b93-083e-412b-a83e-114497453464 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.433583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.435088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.440960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 325/641] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:04.440960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e27b93-083e-412b-a83e-114497453464 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.545742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e27b93-083e-412b-a83e-114497453464 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.546739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 317/642] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:04.547442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.584499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.585105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.585385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.585657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.586323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:04.586626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.588015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.590205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.598188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 326/643] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:04.599001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.632105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.659768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bed8a1-6df8-4b7e-b3c9-4109cec3c293 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.696654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.697500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.697950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.698362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.706280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f853-34d0-47df-ba44-ac6aa96281d3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.709261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:04.714107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:04.715177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:04.716022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 318/644] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:04.716718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.747678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.748273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.748606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.748887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.818404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.820910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.822970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.827419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9f4b4-7bff-4470-815f-bd1c3d11464f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:04.863412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:04.864451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:04.864734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:04.865009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:04.867718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:04.879539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:04.881484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:04.899796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:04.900923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:04.907246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:04.907246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:04.907246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:04.907246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 319/645] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:04.907246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:04.928447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 327/646] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:04.929351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.014881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341780a9-b698-4ca4-8088-97bd44961052 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.039070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.044349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71861-4272-4978-8210-4154cfec06e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.051757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.052797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.053082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.053358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.066101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.068487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.079243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.079840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.080120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.080390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.081049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.081349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.082710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:05.087253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:05.088215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:05.089101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:05.089101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.089101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:05.091530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 320/647] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:05.092508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.093281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 328/648] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.094190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.141190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.207950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172e93d-dee6-43ce-bcea-84d965a6d22d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.209011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e53a-1c09-41b2-b652-3ea4775f910c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.243600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.244332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.244611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.244879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.245463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.246382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.246663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.246927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.247609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.247910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.249356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.257334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.258784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.261496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 321/649] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.267928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:05.275912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:05.276874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:05.277790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:05.277790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.277790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:05.277790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 329/650] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:05.278023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.296417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.373869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.378076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884dca-c62d-46d3-8924-c9786619a52c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.380257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6ef0d-caee-4d7d-8d28-8c8f69796c81 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.412014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.412642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.412924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.413195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.413339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.413946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.414224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.414489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.415159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.415459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.416826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.423481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.424923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.428504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 330/651] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.429290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:05.440194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:05.441303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:05.442395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:05.442395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:05.442395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:05.442395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 322/652] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:05.442395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.501571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.531880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11dc5-d5f5-4094-8d3d-8861d68cf7b2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.544693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dbdf7-84ad-4b31-9f6a-7c19625fb8c7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.566337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.566972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.567281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.567730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.575310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.577111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.577547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.578154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.578431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.578698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.579369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.579664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.581013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-673dab66-f1b3-40d0-9949-426fb9caa892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:05.588256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:05.589239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:05.590158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 331/653] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:05.590774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd628-c843-4eb0-b618-ee5cb3a9302e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.591068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd628-c843-4eb0-b618-ee5cb3a9302e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.591365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fd628-c843-4eb0-b618-ee5cb3a9302e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.592652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 332/654] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:05.596729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fd628-c843-4eb0-b618-ee5cb3a9302e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.596826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 323/655] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.596826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.608548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.698859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ef81-6c54-4e7a-9619-ace77f36cb44 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.701340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fd628-c843-4eb0-b618-ee5cb3a9302e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.729712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.734162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.734771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.734893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.735078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.735355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.735737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.736003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.736129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.736306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.736519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.737686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.744005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.745447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.749906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 324/656] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.750726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0428e48b-efcb-49fa-87d7-1331e411004b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:05.757573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:05.758688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:05.759765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 333/657] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:05.760345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818629-667b-405c-9538-ad62985da251 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.760345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818629-667b-405c-9538-ad62985da251 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.760345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818629-667b-405c-9538-ad62985da251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.778244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3a818629-667b-405c-9538-ad62985da251 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:05.779456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a818629-667b-405c-9538-ad62985da251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:05.823214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:05.844077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 334/658] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:05.844905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818629-667b-405c-9538-ad62985da251 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.869395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4370215-6aad-48a1-92ca-7ddd6e387d19 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.904068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:05.904694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:05.905000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:05.905254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:05.905933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:05.906229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:05.907738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:05.920781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 325/659] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:05.921573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:05.962014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818629-667b-405c-9538-ad62985da251 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:05.988984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.002215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.002838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.003145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.003416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.010528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.012158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:06.023417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:06.024344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:06.025218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 335/660] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:06.025690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.034539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceb73f-5bf4-4631-8a6a-56b368650f30 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.065307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.073308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.074286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.074562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.074829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.084336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.085913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:06.097559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:06.098463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 326/661] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:06.099503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.138942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64969f70-cda7-41b8-baab-c75721e40030 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.171686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.172270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.172844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.172844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.173508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.173812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.175255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.192512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 336/662] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:06.193620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.207874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c120d-1b15-40bf-bf47-b163edf1f58d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.237923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.244172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.244835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.245114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.245384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.246500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.246808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.248290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.254549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 327/663] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:06.255368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.315167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e10005-bf92-4841-a2ce-da4bab03b20d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.322456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.363831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.364423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.364955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.364955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.365762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.365937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.367458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.375548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ead14-3eb2-4649-8d3e-df4fd156b089 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.386854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 337/664] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:06.386854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.418322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.418928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.419230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.419500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.420345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.421344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.422065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.440842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 328/665] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:06.441794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.505853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.508323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70973053-2d33-4479-a207-a952f65c10c2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.545549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.546170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.546442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.546706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.549730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c339b38-7f23-4738-bcc7-1529f94e2e4e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.557044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:06.562267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:06.563449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 338/666] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:06.564697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.566174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.594179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.594874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.595196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.595473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.675309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.677319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.677844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feb074-3db1-4699-8212-2c8453cfbb67 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.715690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.716454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.716824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.717193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.718011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=a77972e7e64f47c7a3fe7153841976c3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.718411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.719948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:06.726626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 339/667] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/role_assignments?user.id=a77972e7e64f47c7a3fe7153841976c3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:06.727717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.756701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.810229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 329/668] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:06.811677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:06.828698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:06.863677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c765dc-2f6f-41fb-a46d-6b3b0dbce54f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.906839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.907612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.907997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.908343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.939785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b436a-5402-485a-9d48-6fb50109da52 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:06.992478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:06.993771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:06.994284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:06.995381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:06.996387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:06.996695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:06.998127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.001066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.002599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.011994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 330/669] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:07.011994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.029581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.106911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 340/670] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:07.112236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.114569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ba5ed-ea76-4ace-8e30-4e16f361c290 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.146209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.151631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.152448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.152891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.153300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.165766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.167468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:07.182509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:07.183716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:07.184834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:07.184834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.184834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:07.186619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 331/671] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:07.187694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.221811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab75c40-90bc-4245-b092-f1f914b1d652 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.248207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.261033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.261862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.262473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.262889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.283986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.283986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:07.299065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:07.300232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:07.301360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:07.301360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.301360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:07.307192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 341/672] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:07.307192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.319217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d5139-d06c-4861-8345-ec2ca565c02a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.351437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.362505 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.363155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.363432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.363696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.378605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.380375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:07.401583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:07.402575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:07.403739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:07.403739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:07.403739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:07.403739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 332/673] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:07.407342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.419270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb648-2aac-47dd-b87c-79a28a66294e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.454667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.455301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.455578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.455846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.456485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:07.456815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.458145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.470624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 342/674] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:07.471500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.511363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.550850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29174e2f-82b1-441d-b038-d5533124bbe8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.581129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1571e8-26e8-4ff1-a758-784b6469d0e2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.596969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.597848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.598133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.598407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.599148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:07.599489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.601283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.622380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 333/675] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:07.622380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.624370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.625185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.625630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.626063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.626887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:07.627388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.628914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.641535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 343/676] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:07.642533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.666129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.731434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a37d9-8efe-4fe5-8a46-dd9b11b28ab6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.758025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.764089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25713085-afb2-4107-888f-9f87dfbad4b3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.778110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.779329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.779715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.779992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.789581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.791878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.802374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.803288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:07.803735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:07.804193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:07.807521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:07.808378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:07.809255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:07.813827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 334/677] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:07.815012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.815735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:07.817351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:07.831998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:07.832976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:07.833896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:07.841307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 344/678] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:07.841307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:07.944550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:07.949402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3fc6-ce03-4e6e-9221-2b13667170c7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.964785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540804-06b5-40e5-af1c-29dd87682162 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:07.998746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:07.999836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.000274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.000761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.003330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.004144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.004625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.005101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.005851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:08.006218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.007742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.013433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.016952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.026826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 345/679] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:08.026826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.043724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:08.045351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:08.046616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:08.047612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:08.055893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 335/680] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:08.056691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.143685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634319c-04ce-465e-a0cf-d6b4014803a7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.186249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.186848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.187466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd78a3-a905-4ed1-9bee-b019ca0f515e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.188289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.188619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.189383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:08.189929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.192390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.219551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 346/681] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:08.219551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.230822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.231883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.232358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.232906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.234275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=9a1ecf64ce544ee5aa7839acde57bea2) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:08.234789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.236620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.243648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 336/682] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/role_assignments?user.id=9a1ecf64ce544ee5aa7839acde57bea2&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:08.250407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.308923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.315186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.349536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7afb-0c6a-43d5-8859-3a9ba6ff9edd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.396262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.397572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.398333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.398779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.399978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:08.401005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.403013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b511-ab0b-4795-b390-7c5348388fc5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.419252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 347/683] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:08.421181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bab72-0f5b-4700-ac94-2f9d6ece187d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.421720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bab72-0f5b-4700-ac94-2f9d6ece187d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.421878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bab72-0f5b-4700-ac94-2f9d6ece187d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.427437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 348/684] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:08.428498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bab72-0f5b-4700-ac94-2f9d6ece187d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.455085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d063a33-f063-4918-b3a4-4629d8591d16 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.505246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.506011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.506297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.506568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.558806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bab72-0f5b-4700-ac94-2f9d6ece187d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.596204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.598299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.600435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.601247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.601525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.601836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.603423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=8d87047620874e09bc5d851978949acc) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:08.603730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.605196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ec9eb-0c07-4020-806e-1c63d31ad55b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.607151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.614065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 349/685] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/role_assignments?user.id=8d87047620874e09bc5d851978949acc&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:08.615922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0381c5-332f-4d4a-b66a-076b19a5dffe None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.616199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0381c5-332f-4d4a-b66a-076b19a5dffe None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.616466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0381c5-332f-4d4a-b66a-076b19a5dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.635912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.639007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8c0381c5-332f-4d4a-b66a-076b19a5dffe None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:08.641024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c0381c5-332f-4d4a-b66a-076b19a5dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:08.765054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 350/686] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:08.766695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab656b-5504-4ce8-8182-3b86db57e770 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.766998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab656b-5504-4ce8-8182-3b86db57e770 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.767282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab656b-5504-4ce8-8182-3b86db57e770 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.773102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 351/687] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:08.775136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.775421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.775692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.781421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 337/688] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:08.791136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:08.816748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:08.817980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:08.901143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb885ee3-48e2-4895-82c0-83433a3ad290 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:08.914502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 352/689] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 03:54:08.927737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:08.940441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:08.941402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:08.941887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:08.942430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:08.953602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:08.955365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:08.970694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:08.971772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:08.972754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:08.972754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:08.972754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:08.974804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 338/690] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:08.980547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.044470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.067123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.105925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bbaf-cf02-4506-8f21-f1eb963d588f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.181143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.181755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.182027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.182292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.182940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:09.183257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.184616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.189783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bf82-4862-4594-8401-f70c193b252f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.201719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 353/691] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:09.202728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.233753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.234385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.234662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.234933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.249318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:09.255828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:09.256808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:09.258158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 339/692] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:09.259554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.301945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.333185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c797e-7e24-4436-b23a-04a37a8d4cb0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.351400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.370083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.370713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.371485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.371727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.375893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f5540-00d0-4344-b759-b5d6bdd16db3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.383174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.384748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:09.400489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:09.401679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:09.402773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:09.402773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:09.402773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:09.402773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 354/693] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:09.403115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.417036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.418017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.418457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.418883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.426590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.428479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:09.441167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:09.442258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:09.443391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:09.449925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 340/694] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:09.450984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.528222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ced1f-271b-4bd4-9bcf-aa2551b624fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.551383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.558865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac91f6d-f088-4b4e-86c6-334b39d582d8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.566405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.567368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.567809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.568235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.569241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:09.569828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.571650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.585811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 355/695] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:09.587039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.600181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.600851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.601135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.601406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.602433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:09.602736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.604801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.620525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 341/696] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:09.622009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.626158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.705892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ae27b-299e-43f8-ba3d-42fa358ab517 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.727678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c419f8-b89f-458c-82f4-715388a21577 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.742337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.742701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.743100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.743376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.744379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:09.744891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.746886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.765704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 356/697] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:09.765704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.770021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.770922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.771436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.771904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.783020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.784714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:09.799958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:09.801014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:09.801935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:09.801935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:09.801935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:09.808517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 342/698] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:09.809691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.819398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.864891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:09.875107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc1102-f2b7-433b-99f1-c7fc0b0177a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.913128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.914210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.914777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.915284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.916128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.917132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7f0f-8442-4127-b828-39836477e773 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:09.918016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.946558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 357/699] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:54:09 2022] POST /identity/v3/users => generated 254 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:09.948222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:09.955114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:09.955763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:09.956102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:09.956316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:09.957171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=b526cee1dc234083a7f4897fee8e22b6) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:09.957476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:09.958980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:09.970704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 343/700] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/role_assignments?user.id=b526cee1dc234083a7f4897fee8e22b6&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:10.122210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57986-7b16-4ef7-a376-071f770bcb92 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:10.168679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:10.169306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.169584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.169855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.170587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:10.170889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:10.172392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:10.185360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 358/701] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:10.191487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:10.275471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:10.315430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48bd14-7dcb-43ea-9c5d-fc174cab1b3a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:10.350731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:10.351395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.351675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.351945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.361623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:10.363246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9968324a-8106-469d-88f1-b3bbf6e95f55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:10.376327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:10.377417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:10.378979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 344/702] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:10.383412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:10.395907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:10.491769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811d99-00d4-4b58-862c-1e50cb76ce7a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:10.531229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0620-697b-4b6e-925d-7fff49fd81f5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.531422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0620-697b-4b6e-925d-7fff49fd81f5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.531668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0620-697b-4b6e-925d-7fff49fd81f5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.533305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 345/703] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:10.538303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.538587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.538861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.546385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:10.546994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.547332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.547609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.548378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:10.551048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:10.551048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:10.572865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:10.574087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:10.580576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 359/704] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:10.586896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:10.687781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 346/705] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:10.704690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:10.754424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b02a-6086-4927-ac5f-299f20c3b62b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:10.788343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:10.788874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:10.789148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:10.789413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:10.876217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:10.877674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:10.941549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:10.983334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 360/706] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 03:54:10.989544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.040424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.065279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.115234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be7148-d679-4a16-bfa2-49a63118bd27 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.155804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750df6-13bc-4da2-aca5-c29307b4e7b9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.161056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.162116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.162596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.163105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.177484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.179836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.190743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.191432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.191674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.191941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.192516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.193855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c45c9d84-1ceb-4220-b384-0b74574279c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:11.199544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:11.200491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:11.201428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:11.201428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:11.201428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:11.203735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 347/707] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:11.204890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:11.205887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:11.206727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:11.208334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1a20-d091-4a08-988c-cd5d091979ff None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.208816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1a20-d091-4a08-988c-cd5d091979ff None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.209280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1a20-d091-4a08-988c-cd5d091979ff None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.210805 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 361/708] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:11.211451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 348/709] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:11.215763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.215889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe1a20-d091-4a08-988c-cd5d091979ff None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.249299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.321010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe1a20-d091-4a08-988c-cd5d091979ff None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.334784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.337573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951e034-7f44-4d73-80c1-18e1b99f35f5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.357778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.358489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.358667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.358912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.359582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:11.359877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.361237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141a4ce-8a04-472d-bac8-16d4808bc0f5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.370362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 349/710] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:11.371960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.372240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.372513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.378945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.379852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.380324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.380806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.381646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:11.382163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.383755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.390941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:11.392171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:11.397361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 362/711] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:11.398493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.474439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 350/712] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:11.474439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.524242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.557068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2e9e5-c9c8-43db-9260-648cd768a596 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.579336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa680e-a80a-43e9-8da9-8fbfb16df1db None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.591390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.616708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.617748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.618031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.618295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.625862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.627558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:11.640229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:11.641200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:11.642156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 351/713] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:11.647431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.670884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.671830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.672349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.672813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.673647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:11.674153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.675821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.689951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 363/714] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:11.801284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfeee1-2d55-433e-9a85-9113b6928b18 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:11.827885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.874746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:11.875806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:11.876294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:11.876757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:11.877685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:11.878208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:11.880060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:11.894804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 352/715] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:11.904827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:11.964648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:11.976810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e43e60-d00d-4362-be91-05520581f435 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:12.014160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:12.014795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.015123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.015396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.015978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:12.017336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:12.026693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:12.027752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:12.028583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 364/716] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:12.035896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:12.083473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:12.128795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e0f2-59dc-4112-943e-31b9a0efab23 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:12.175741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e3925-64bd-4851-8454-070057c626df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:12.176351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3925-64bd-4851-8454-070057c626df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.176624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3925-64bd-4851-8454-070057c626df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.176889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e3925-64bd-4851-8454-070057c626df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.177621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3925-64bd-4851-8454-070057c626df None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:12.177925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3925-64bd-4851-8454-070057c626df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:12.179395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e3925-64bd-4851-8454-070057c626df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:12.188539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad2eda-5efa-4e04-b45a-c35574509434 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:12.194151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 353/717] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:11 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:12.242268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:12.243303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.243802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.244268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.245148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] RBAC: Adding query filter params (name=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:12.245675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:12.247772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:12.259709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fcc1e-81e0-4e81-86a5-c3ede01d392d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.260045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fcc1e-81e0-4e81-86a5-c3ede01d392d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.260258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99fcc1e-81e0-4e81-86a5-c3ede01d392d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.260837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 365/718] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:12.261744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 354/719] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:12.273097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:12.276550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870d135-f7c0-4358-8f33-97b443fd7e1b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.276802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870d135-f7c0-4358-8f33-97b443fd7e1b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.277065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870d135-f7c0-4358-8f33-97b443fd7e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.323487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1870d135-f7c0-4358-8f33-97b443fd7e1b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:12.325056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1870d135-f7c0-4358-8f33-97b443fd7e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:12.402831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:12.423467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9de2c-a1b6-4569-905b-b974646eb308 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:12.436973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 355/720] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:12.460417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:12.461028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.461304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.461573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.462262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:12.462616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:12.464072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462e09-be55-40e4-99ba-eb81440cf9fe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:12.469538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 366/721] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:12.645565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c0d0-b108-4531-8782-4d2f42aec6d1 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.646121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c0d0-b108-4531-8782-4d2f42aec6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.646121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c0d0-b108-4531-8782-4d2f42aec6d1 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.647738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 356/722] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:12.652660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.653137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.653222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.677302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:12.678518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:12.757737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 367/723] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:12.808891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58c0d0-b108-4531-8782-4d2f42aec6d1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:12.834598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:12.939175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58c0d0-b108-4531-8782-4d2f42aec6d1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:12.976325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:12.976968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:12.977246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:12.977513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:12.989712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:12.994925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:12.996257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:12.997111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 357/724] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:13.003049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.129117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:13.157047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884db9ad-863c-460d-a404-68b47bf08a52 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.192020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.195175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.195841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.196121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.196389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.197053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:13.197346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:13.198702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:13.207837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 368/725] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:13.216297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.311779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:13.334855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d6dcb-4871-43f9-8d34-2d5f00c30f25 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.346573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb3005-ca42-44c5-93bf-cb4d43b32531 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.407391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.408027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.408303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.408572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.410567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.411222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.411492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.411756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.424093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:13.433564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:13.435047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:13.436410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:13.441192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:13.442352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:13.443227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:13.444173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 358/726] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:13.450196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:13.468044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:13.469305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:13.470230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:13.470230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:13.470230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:13.470230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 369/727] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:13.475662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.647784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cafec-9930-4f91-ac9d-2ec5440bbf65 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.651827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a410-d306-4ed0-871d-392fde2a370c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.708872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.709399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.709738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.710009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.710677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:13.710973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:13.712493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:13.721126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.721729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.721999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.722262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.722907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:13.723413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:13.724764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:13.728018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 359/728] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:13.733677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.743337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:13.753007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 370/729] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:13.758227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:13.848270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:13.889804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaa28b-0eeb-4d0c-ae54-84ccec3721ba None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.944335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe280b-19be-49c4-95d3-96ef2d358f9e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:13.954083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.954742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.955019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.955325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.966555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:13.968109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fe5dd2af-2854-4d86-b3d2-c0e554934315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:13.983894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:13.984846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:13.985728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:13.985728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:13.985728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:13.986996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:13.987778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 360/730] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:13.989102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82b7eb-9a2d-41e6-9f2c-71a687a66223 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.990000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82b7eb-9a2d-41e6-9f2c-71a687a66223 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.990415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82b7eb-9a2d-41e6-9f2c-71a687a66223 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.991761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:13.992388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:13.992825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 361/731] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:13.994912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:13.997496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82b7eb-9a2d-41e6-9f2c-71a687a66223 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.009741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.011625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-315a2f98-a61e-463f-a142-61a38ff4eb96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:14.026759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:14.027756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:14.028683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:14.031164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 371/732] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:14.032612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.033143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.033777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.060400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:14.061987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:14.067560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.119578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82b7eb-9a2d-41e6-9f2c-71a687a66223 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.143464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 372/733] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:14.144409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.165170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.166235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.166703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.167242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.168222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:14.168897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.170855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.184658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 362/734] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:14.195596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.205625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.261890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4310f3-8e26-4850-9cd6-e0a8fc949ca7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.327101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.327974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.328426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.328851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.329750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:14.330987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.332627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.372480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 373/735] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:14.374942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.401401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc5e1a-0392-483c-b953-b3508181ab8c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.429776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.451977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.452706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.452985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.453255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.461121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.462569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5b310d3f-3b9b-44e1-8fcb-cbc360446d01 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:14.475644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:14.476842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:14.478106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 363/736] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:14.478747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17838aa-770e-46da-97be-6eb6de04713b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.478747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17838aa-770e-46da-97be-6eb6de04713b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.478747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17838aa-770e-46da-97be-6eb6de04713b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.480333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 364/737] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:14.485195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17838aa-770e-46da-97be-6eb6de04713b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.490550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.493755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca52181-c45c-4e16-9184-574a6960ed1b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.530293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04617f10-201d-4c52-9364-18316f771efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.530809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617f10-201d-4c52-9364-18316f771efe None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.531114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617f10-201d-4c52-9364-18316f771efe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.531387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617f10-201d-4c52-9364-18316f771efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.598637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17838aa-770e-46da-97be-6eb6de04713b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.608117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04617f10-201d-4c52-9364-18316f771efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.610466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04617f10-201d-4c52-9364-18316f771efe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.635900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.636506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.636783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.637057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.637789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:14.638096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.639815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bbdce-277a-45e4-a64d-b18f579f0ee7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.658799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 365/738] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:14.660024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.660306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.660576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.679841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:14.681043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:14.710021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.716962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.726545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 374/739] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:14.727465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04617f10-201d-4c52-9364-18316f771efe None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.757730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 366/740] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:14.758584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.839876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04617f10-201d-4c52-9364-18316f771efe None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.872927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.873719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.874135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.874409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.875113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:14.876486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:14.877338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8808-d6a2-43ab-bbe4-1b522b2c3a8f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:14.895580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 375/741] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:14 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:14.896377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:14.931675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:14.932965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:14.933543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:14.934036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:14.966542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:14.968640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.018401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e25839-7468-4d0b-a180-2706b522259b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.020642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.022517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.058264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.058893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.059224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.059493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.071347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.072792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-16b9176e-152c-4fce-aacf-ce0ac4343a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:15.089641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:15.090661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:15.091611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:15.091611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.091611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:15.091611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 376/742] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:15.092527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41226c-619e-4a3f-b17b-ca6872259ff3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.092527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41226c-619e-4a3f-b17b-ca6872259ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.092821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41226c-619e-4a3f-b17b-ca6872259ff3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.097868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 377/743] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:15.097868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41226c-619e-4a3f-b17b-ca6872259ff3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.142232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 367/744] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:14 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 03:54:15.142950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.205727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.214451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41226c-619e-4a3f-b17b-ca6872259ff3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.227236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.250184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c76090-3f22-4e50-93c5-1180c897adcd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.254600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.255256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.255531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.255797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.271750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b271816c-3fb0-45dc-9ed7-dedac697120f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:15.278304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:15.279262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:15.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:15.280977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 378/745] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:15.282090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.282369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.282641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.290941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.291660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.292006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.292341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.293055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:15.293418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.294858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.302721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:15.303949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:15.311816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 368/746] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:15.311816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.374449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 379/747] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 03:54:15.375606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.425948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d5ddf-2581-425a-9c2a-7525abf14c76 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.444593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.465214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.466015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.466499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.466950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.478584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.480261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:15.495622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:15.496889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:15.497783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:15.497783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.497783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:15.497783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 369/748] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:15.498646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.512376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.516776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4857de-f595-4888-bd9c-38af6f18b1a9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.555390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.556439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.557042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.557525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.558426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:15.558916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.560729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.570930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 380/749] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:15.572014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.612962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741d43-168a-4b90-9ff0-0d2cc457f142 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.647306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.648198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.648198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.648370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.655463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.656970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:15.669116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:15.670957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:15.671888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:15.675154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 370/750] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:15.675997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.684811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8df1b-b56a-4059-9b30-0594fd77f7cb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.707726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.720130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.720729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.721008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.721278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.721911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:15.722205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.723590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.742135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 381/751] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:15.742135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.770203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.782367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995fa3-97a3-4f03-93d7-dd009efc07cd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.817169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.817794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.818368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.818368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.828997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.830407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:15.844671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:15.845623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:15.846536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:15.846536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:15.846536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:15.846536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 371/752] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:15.846778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.860725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2850f-fee6-46fd-adec-4a6ebd038d42 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:15.897496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:15.898521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:15.899159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:15.899746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:15.900588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:15.902376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:15.919648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 382/753] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:15 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:15.920675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:15.937839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:15.962315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41b13-ffc5-4974-b06d-9591235ffe80 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.022083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.022698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.022976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.023300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.024279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.024585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.026012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.028157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.045967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 372/754] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.047216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.051159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef53c1-fa5e-44e1-8a06-5eeb8d424299 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.088619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.089774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.089774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.090229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.097775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.099362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:16.112312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:16.113440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 383/755] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:16.114446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.174369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d89bbb-bfdd-4046-890d-2e7d997127f9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.211459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.212060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.212337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.212613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.213257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.213555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.214930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.222662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.232267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 373/756] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.233617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.242643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6782c-ed3a-443c-9d17-ba0104ae7e53 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.279956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.280822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.281101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.281370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.282148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.282454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.286185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.299053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 384/757] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/role_assignments?user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.299053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.318232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.425430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e0eb-1cdb-409d-a7fc-cbac9d0ff732 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.490815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.491457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.491736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.492065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.492770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.493121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.494571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.508883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f49073-778d-43ea-8de2-6621fbc973b1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.511820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 374/758] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.524066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.557808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.558822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.559327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.559792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.569902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.571668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:16.586297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:16.587320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:16.588270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:16.591448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 385/759] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:16.596943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.664823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.675275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.716501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd309842-91c5-4e75-8a78-480ff4ada3e3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.755707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.757440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.757930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.758404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.759369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.760134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.761750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faf863-bb84-4c19-a644-6bc0c1897a5f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.762268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.783417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 386/760] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.788718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:16.833486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.834470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.836494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.836965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.837925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=b526cee1dc234083a7f4897fee8e22b6) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:16.838520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:16.841421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a6ff6-5bd0-443e-800b-db09e5f784a9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:16.852688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 375/761] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/role_assignments?user.id=b526cee1dc234083a7f4897fee8e22b6&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:16.854990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79bd1-164d-4957-aa20-7cbef81b5e27 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.856151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79bd1-164d-4957-aa20-7cbef81b5e27 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.856803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c79bd1-164d-4957-aa20-7cbef81b5e27 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.859704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 376/762] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:16.880823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.881452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.881966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:16.912332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:16.913562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:16.948786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20207135-4d58-4c16-808c-3a0b849ab271 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:16.976844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:16.991678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:16.992478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:16.992751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:16.993015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.020871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.027051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:17.044189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:17.045246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:17.046124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:17.046124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:17.046124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:17.046124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 387/763] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:17.046702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 377/764] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:17.052028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.194727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf280ea-6fe4-4f05-8f9d-a901b3b8c493 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.242117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.243082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.243621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.244082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.244958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:17.245442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.247340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.266173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 388/765] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:17.272132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.339905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.361066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:17.405965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:17.439258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04a7d1-0ece-4055-badd-bfb6514b539e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.480494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.481126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.481402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.481728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.490036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.491520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:17.505910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:17.506883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:17.507781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 378/766] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:17.516045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.520312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5243450-22b8-4223-a453-1dde17fafac1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.555800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.556435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.556705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.556971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.557549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.558899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:17.568325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:17.570550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:17.572585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 389/767] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:17.577899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.676792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:17.680109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60fcc0-b8a5-4296-9a21-0670e43c525e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.719751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.720364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.720634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.720904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.721536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:17.721829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.723334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.728562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 390/768] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:17.734468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.764968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79bbce-bf51-42a1-a2b0-3758b8f3c20d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.790910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:17.848116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68841b-bc08-41d2-922b-8a27f3579cdd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:17.858735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.859526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.859899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.860355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.861053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:17.861449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.862984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.886285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:17.887271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:17.887720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:17.889012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:17.889137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 379/769] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:17.889932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:17.890396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:17.891925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:17.894687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.898012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 391/770] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:17.904609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:17.935692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:18.018399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc614f-3173-4b72-991c-15d4d3cc3c57 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:18.056238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:18.057175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.057623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.058057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.058920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] RBAC: Adding query filter params (interface=public, service_id=230926d7850741098e8aef721bef86af, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:18.059407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:18.061868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:18.071589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 392/771] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:17 2022] GET /identity/v3/endpoints?service_id=230926d7850741098e8aef721bef86af&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:18.078156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:18.127783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac86684-a2bd-4ad2-b066-5e11d683857e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:18.162910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:18.163612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.163890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.164161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.166093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=ed122ea299374f9cb2b8b4854df14d65, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:18.167646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92c668-bfd7-4b26-b264-0cfe62cbfc32 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:18.186840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 380/772] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:17 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:18.188349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe80-72b0-4ac0-af97-1442481db458 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.188634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe80-72b0-4ac0-af97-1442481db458 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.188909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1efe80-72b0-4ac0-af97-1442481db458 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.189799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:18.191971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 381/773] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:18.196986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e784-56bf-4156-b9df-330ed73bb0ee None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.197266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e784-56bf-4156-b9df-330ed73bb0ee None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.197529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e784-56bf-4156-b9df-330ed73bb0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.205846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb34c2-779c-416c-a271-749f3599f253 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:18.224463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bd71e784-56bf-4156-b9df-330ed73bb0ee None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:18.226958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bd71e784-56bf-4156-b9df-330ed73bb0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:18.270301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:18.282133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:18.283327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.283327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.283327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.283938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:18.285785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:18.297999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 393/774] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:18 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:18.319069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 382/775] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:18.637730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:18.662781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75f1a-5c28-433a-95ba-2e5e8330b252 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.663140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75f1a-5c28-433a-95ba-2e5e8330b252 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.663373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75f1a-5c28-433a-95ba-2e5e8330b252 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.665212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 383/776] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:18.670433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.670710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.670975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.688948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:18.690107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:18.785408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 384/777] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:18.792299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c237-0117-4053-8d70-b633c5d9fd91 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:18.828671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:18.829677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:18.830219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:18.830682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:18.831486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:18.833079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:18.846026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:18.847053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:18.847889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:18.851445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 394/778] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:18 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:18.857732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:18.973062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:18.981535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbe808-6c11-480c-b05b-0f975f8caf1f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.018053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.018811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.019212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.019571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.020295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] RBAC: Adding query filter params (name=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:19.020687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.022197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.027240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 385/779] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:19.033019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.062953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:19.134961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.157212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574e46b-d671-406f-9cb4-cd32ba1804ae None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.196597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.197214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.197510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.197766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.198402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:19.198705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.200888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.207151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 395/780] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:19.217155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.289759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9531e62-c0d7-4061-976a-7917d75aaa75 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.328753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.329668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.330082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.330489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.349370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:19.354422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:19.355343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:19.356161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 386/781] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:19.363297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.400298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:19.424677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:19.442208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16ec4-e0dc-4726-b4bc-76695f5f1049 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.481592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.482201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.482529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.482797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.483537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] RBAC: Adding query filter params (interface=public, service_id=f81dd92dab0f4ea599143877daaa51fd, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:19.483840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.485428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32281140-451e-4121-a126-f71ca4ef0d03 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.491291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 396/782] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/endpoints?service_id=f81dd92dab0f4ea599143877daaa51fd&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:19.492641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806d562-733c-414c-8ceb-f91f9fb4a6e7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.492920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806d562-733c-414c-8ceb-f91f9fb4a6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.493187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806d562-733c-414c-8ceb-f91f9fb4a6e7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.494378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 397/783] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:19.496322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b140098-333f-4a17-bbd2-6c409cc0de28 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.499618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806d562-733c-414c-8ceb-f91f9fb4a6e7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.539012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.540078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.540558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.541014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.541863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:19.542360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.544448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579293-6cf6-49ea-b1e9-eea5ccc49d06 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.555794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 387/784] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:19.557203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.557673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.558661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.587048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:19.587048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:19.611238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806d562-733c-414c-8ceb-f91f9fb4a6e7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.647259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.647865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.648200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.648472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.649074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.650546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.655659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 388/785] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:19.656519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.665012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 398/786] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:19.666980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:19.768292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f9c8-b164-4392-804c-781de26dcfbd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:19.807062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:19.808383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:19.808836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:19.809251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:19.821296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:19.823054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:19.839717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:19.840907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:19.842124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:19.842124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:19.842124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:19.843974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 389/787] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:19.853398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:19.876077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:19.980362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33d448-6925-4147-ab61-d53af14987c3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.015489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.016097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.016374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.016643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.017285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:20.017581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.018954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.030900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 399/788] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:20.037269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.050630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.115913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.159948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.197296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafd672-a0df-4542-ae30-608fbc643328 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.222459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ca56-5cfd-4189-9a41-d271f165a56f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.274903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.277161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.278567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.279085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.300553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.301658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.302127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.302585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.304025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.306908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.324025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c4dffc86-04e3-4158-baa8-728a3c022c1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:20.329537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:20.330460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:20.331448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 400/789] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:20.332633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c7323-fbdc-4d97-9184-228797782ebf None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.332633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c7323-fbdc-4d97-9184-228797782ebf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.332801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c7323-fbdc-4d97-9184-228797782ebf None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:20.333062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:20.333925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:20.334826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:20.337237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 390/791] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:20.337799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 401/791] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:20.338515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.343472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.343472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.343472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.359652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:20.360850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:20.438998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 402/792] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:20.439989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.473503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55397bdb-0e86-440a-aef9-5773959de609 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.500988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.510042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92700b-5116-452b-896e-751123b1619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.510661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92700b-5116-452b-896e-751123b1619c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.510946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92700b-5116-452b-896e-751123b1619c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.511252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92700b-5116-452b-896e-751123b1619c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.512544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92700b-5116-452b-896e-751123b1619c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:20.512544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92700b-5116-452b-896e-751123b1619c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.513761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92700b-5116-452b-896e-751123b1619c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.523427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 391/793] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:20.528972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92700b-5116-452b-896e-751123b1619c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.556459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.605450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef5869-3b75-4c14-85a6-34aaa920807c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.640903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.641830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.642298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.642851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.643823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:20.644521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.646144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.664349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 403/794] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:20.670227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.701112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92700b-5116-452b-896e-751123b1619c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.773100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3befc8-3a9e-42df-9020-00308b39bea2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.778491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.779254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.779534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.779799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.792684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.794849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.798094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.813681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.814633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.815147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:20.815251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:20.816451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:20.817310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:20.817310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:20.817310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:20.817310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 392/795] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:20.817538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.818442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.829958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:20.831738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:20.848342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:20.849288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:20.850171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:20.850171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:20.850171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:20.852213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 404/796] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:20.856740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:20.858736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:20.934110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4010-ad14-43b8-bd4a-2547bc20f5b8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:20.971449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:20.971984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:20.972257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:20.972482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:20.984720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:20.991296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:20.992235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:20.993051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:20.997651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 393/797] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:20.998285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.086849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e204d9-d810-4653-a024-49da21f5e39f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.104869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.113085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c3fc2-4e1f-4bcb-9f59-64b0f7ace6d0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.123642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.124499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.125024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.125435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.126213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.126662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.128194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c75ff3-3430-4c8e-8a34-be1f9596db2d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.141056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 405/798] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.142309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc75b-b0af-4b8d-b341-236187b061dc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.142593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc75b-b0af-4b8d-b341-236187b061dc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.142861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecc75b-b0af-4b8d-b341-236187b061dc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.144208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 406/799] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:21.144791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc75b-b0af-4b8d-b341-236187b061dc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.157222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.157835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.158119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.158394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.159073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.159381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.160795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dba848-0fad-402e-bbe3-8c02d73e600e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.173308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 394/800] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.174474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.174757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.175050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.183882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.192904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:21.194094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:21.261182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecc75b-b0af-4b8d-b341-236187b061dc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.262043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 395/801] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:21.271148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.311990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.312600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.312878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.313150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.314271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.314573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.316627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.328107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 407/802] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.329431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.370399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.392213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395d80f-e17a-4e81-8edb-531f0bf89d0d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.437275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.437592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.438166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.438564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.438898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.445846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa935ae-4163-4b21-a972-1b67d2cc7fc9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.447779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.449422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:21.462675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:21.463694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:21.464610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 396/803] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:21.465180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.496271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.496900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.497444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.497717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.507696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.509143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:21.525060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:21.526308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:21.527955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 408/804] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:21.529047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.582452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d79bc-bcdf-47e1-b690-4701797e6de8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.618991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.619643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.619923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.620198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.631519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.632976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.635627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.636801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33b80d-9c14-4a00-b1f3-5158b56e4d06 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:21.648742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:21.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:21.650582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:21.650582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:21.650582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:21.650582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 397/805] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:21.650819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.679364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.679968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.680244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.680511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.681242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.681537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.683057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.705171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 409/806] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.705171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.717227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.766685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9b3a9-bbd6-4c01-9ad8-d8926b9dfd30 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.801892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1b98e-1342-445a-9585-e597ea15880e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.803556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.804264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.804542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.804809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.805514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.805812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.807271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.824996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 398/807] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.824996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.841617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.842525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.842790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.843110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.843688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.845035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:21.853849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:21.854959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:21.856249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 410/808] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:21.857045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.887851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.932354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4681-7060-45a0-ad66-4e7f075402b4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.935172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:21.958174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f7f4-1928-4aa7-8ee7-491f09d4777f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:21.965264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.965869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.966148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.966426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:21.967127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:21.967427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:21.968795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:21.980772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 399/809] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:21.982100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:21.995207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:21.996147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:21.996425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:21.996692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.007711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.009150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:22.022923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:22.024058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:22.024928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:22.024928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:22.024928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:22.024928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 411/810] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:22.024928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.083950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c70c6-0cd0-4e30-a609-cf4d72f3a8a5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.100153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.118796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.119477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.119751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.120021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/5eeb4f0ca36d40e4ad8d5b77cdf35068/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.122955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b630c-71fc-4576-9543-d5579d4b987e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.135247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=5eeb4f0ca36d40e4ad8d5b77cdf35068, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.136673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4936f-b587-4f3d-8cb3-c1d9d1dd3d67 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.148324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.150306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 400/811] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:21 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/5eeb4f0ca36d40e4ad8d5b77cdf35068/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:22.162677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4627624-7160-4bf5-abce-ea8a8f541343 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.162677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4627624-7160-4bf5-abce-ea8a8f541343 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.162677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4627624-7160-4bf5-abce-ea8a8f541343 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.162677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 401/812] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:22.162677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4627624-7160-4bf5-abce-ea8a8f541343 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.166402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.167014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.167309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.167573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.168215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:22.168509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.170163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.180361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 412/813] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:22.180361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.311681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8037aec-0ca1-417b-bf65-f77c5bb3a877 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.352046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.352577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.353002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.353357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.353976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:22.354317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.355885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2b3c0-e8ba-48cb-929a-110c2faff15d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.371911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 413/814] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:22.371911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.371911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.371911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.389538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:22.390713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:22.407836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4627624-7160-4bf5-abce-ea8a8f541343 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.439428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.444374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.445081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.445419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.445747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.453827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.455321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:22.467159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:22.468262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:22.469188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:22.471392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 414/815] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:22.472615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.472857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 402/816] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:22.473827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.503246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.583180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543397-ffb3-46a8-ac60-afa02ab168fb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.611712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fcd53-8fce-45f9-9a4c-eb2e62fc7c35 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.623758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.624875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.625325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.625870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.634761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.636951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.646943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.647687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.648050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.648477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.649200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:22.649654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.651318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:22.652368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:22.653392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:22.654611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:22.657249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 403/817] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:22.658281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.664111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 415/818] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:22.665101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.722853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.760600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.776467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaccf0-2340-4c3f-ad40-bf2ab5085412 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.787093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54406d1-3711-4b8b-b6cf-9ce8b3136724 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.812666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77842012-4704-415e-8a52-7a69200af647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.813270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77842012-4704-415e-8a52-7a69200af647 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.813556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77842012-4704-415e-8a52-7a69200af647 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.813809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77842012-4704-415e-8a52-7a69200af647 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.814540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77842012-4704-415e-8a52-7a69200af647 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:22.814840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77842012-4704-415e-8a52-7a69200af647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.816302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77842012-4704-415e-8a52-7a69200af647 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.824782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.825386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.825664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.825953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.827313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:22.827618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.829037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.835354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 404/819] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:22.836740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77842012-4704-415e-8a52-7a69200af647 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.845227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 416/820] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:22.846361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:22.942828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77842012-4704-415e-8a52-7a69200af647 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.949047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aef1-801f-4120-997c-f1dcc3aaf353 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:22.956141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:22.978799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.979440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.979732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.980019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.980660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.982019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:22.985594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:22.986228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:22.986503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:22.986770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:22.987382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:22.988733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:23.000496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:23.001583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:23.002402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 417/821] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:22 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:23.003707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.008082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.014585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 405/822] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:23.016668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.126056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ec922-7d93-49d8-ae99-42b3239e4fb7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.144412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf03c4-91cd-4b7c-8d14-cd99e52ca765 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.163731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.164235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.164579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.164885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.190886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.191605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.191885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.192152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/b526cee1dc234083a7f4897fee8e22b6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.194099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=b526cee1dc234083a7f4897fee8e22b6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.195610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.213993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 406/823] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:23 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/b526cee1dc234083a7f4897fee8e22b6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:23.214507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.215336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.275386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.277306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.282630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.330618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc235-18bd-4e06-baa7-ab70e0c89f98 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.367495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.368103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.368377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.368693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.369322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] RBAC: Adding query filter params (name=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:23.369622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.371341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.376892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 407/824] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:23.382583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.402729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 418/825] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:23.408936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.488781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b755-f402-49a7-800a-243096739a90 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.491384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.510585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.525722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.526331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.526658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.526928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.527592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] RBAC: Adding query filter params (type=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:23.527889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.529446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.532460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648b7d-153b-490a-ad66-d29b7bc63a57 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.536749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 408/826] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:23.542513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.569869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.570977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.571494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.571924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.584052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.585724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:23.602058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:23.603248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:23.604541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:23.604541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:23.604541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:23.606429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 419/827] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:23.612517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.654679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd9c0-a8ef-47ef-9795-63889724a845 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.708897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.709514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.709790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.710060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.713597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.714976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.746129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c25c-b08e-4f4b-a9fd-6d843b70ae3c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:23.756732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 409/828] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:23.787149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.820759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:23.821387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:23.821665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:23.821935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:23.822588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:23.822890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:23.824299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:23.837723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 420/829] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:23.848258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:23.902031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:23.984444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa98a4-9a1e-4427-b209-144df5b06b2c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:24.036038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:24.036599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.036875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.037142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.048300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:24.050165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5f5ecf5e-385c-46b1-9680-4c7349ed74df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:24.063255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:24.064193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:24.065116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:24.067655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 410/830] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:24.076359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:24.214960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0692d-b535-462c-ab8b-6801e3f96701 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.215600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0692d-b535-462c-ab8b-6801e3f96701 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.216042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0692d-b535-462c-ab8b-6801e3f96701 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.217903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 411/831] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:24.221114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93951c4-eac6-46d3-962d-0f580cf24251 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:24.223066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.223276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.223525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.248532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:24.248532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:24.260760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:24.261495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.261839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.262181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.262945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:24.263356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:24.264994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:24.284276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 421/832] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:24.289875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:24.340713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:24.350229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 412/833] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:24.406687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cebc445-035a-47df-adda-2a8bbdb1e7e7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:24.442440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:24.443058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.443365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.443635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.444359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:24.444660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:24.446135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:24.452428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 422/834] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:24 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:24.632638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:24.663971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:24.736022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e5a29-7424-492b-831c-3c0948ca3ae3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:24.773954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:24.774869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.775336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.775752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.786401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:24.791458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:24.792589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:54:24.793671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:24.796109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 413/835] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:54:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:24.802124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:24.904947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43db55-f796-4c3a-8443-6006c457922e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:24.906429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:24.939709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:24.940309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:24.940585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:24.940852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:24.941504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:24.941801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:24.943180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:24.952118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 423/836] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:54:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:24.958305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.054265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:25.064694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40a743-b02e-4172-bf0d-a50ceda6b6de None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.100127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.101109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.101609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.102091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.113197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.114917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8047c062-cbbb-4d65-b087-339860b80ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:25.130689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:25.131691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:25.132573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:25.132573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:25.132573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:25.134627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 414/837] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:25.141021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.229244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c54de-0e8e-4ccb-b097-0e41bd93232a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.229684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c54de-0e8e-4ccb-b097-0e41bd93232a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.230043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c54de-0e8e-4ccb-b097-0e41bd93232a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.231807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 415/838] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:25.237754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.238200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.238615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.252047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7168aae-2ccb-418c-855a-fc697a96aaed None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.258273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:25.259685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:25.300143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.300760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.301098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.301503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.302327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:25.302629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.304501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:25.324261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 424/839] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:25.327426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.358146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 416/840] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:25.360479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:25.434611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d37e1-d085-44c3-894d-6be8f6cdf68d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.468195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.469167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.469633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.470119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.477508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.479170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-133163f5-280c-4c37-89a4-690df45a779d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:25.491282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:25.492235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:25.493168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:25.496039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 425/841] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:25.501963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.579953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:25.612803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7929387-a36e-410f-8388-57c85b23de85 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.637730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4a4f-988b-45c0-8de2-bc6ac2b76463 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.638104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4a4f-988b-45c0-8de2-bc6ac2b76463 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.638269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4a4f-988b-45c0-8de2-bc6ac2b76463 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.639864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 426/842] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:25.640628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a4a4f-988b-45c0-8de2-bc6ac2b76463 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.651838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.652809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.653095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.653367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.655132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:25.655448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.656946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a474-46f6-46d6-ad0d-c64fc185e19c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:25.676063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 417/843] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:25.677404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.677686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.677953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.700533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:25.701772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:25.738095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:25.757862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a4a4f-988b-45c0-8de2-bc6ac2b76463 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 418/844] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:25.770748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.814478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.815722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.816233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.816922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.818179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.820297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:25.830902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:25.834037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:25.835017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:25.835892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:25.837678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 427/845] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:25.842075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:25.890662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06371d8f-9085-49b1-8a03-dfde5ae67723 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.938194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.939448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.940286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.940767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.956334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82ad0-110a-44a7-9d7c-ddc1fbf1c3d3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:25.993852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:25.994459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:25.994732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:25.994999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:25.995654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:25.996311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:25.997755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.007690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 428/846] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:26.010485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.030916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.038506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.040085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.099605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.123398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3560-aec9-4042-8eaf-e8ba649485be None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.150844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 419/847] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:26.150844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.166282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.166891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.167332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.167690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.168391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:26.168689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.170162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.175079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 429/848] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:26.175823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.262716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.292086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f0d2-15d5-4dfc-9a0a-f7fd3bf23bd0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.302086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a57fcc-4575-4159-9672-e7ada600c15b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.330804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.333422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.333935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.334383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.342192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.342920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.343301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.343632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.346827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.348393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.355284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:26.360382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:26.361504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:26.362557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:26.364977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 420/849] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:26.369731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:26.370660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:26.371956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:26.371956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:26.371956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:26.371956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 430/850] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:26.375070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.375251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.401247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.481958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f54a5-959d-4f4f-97bb-b01b70ff674b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.501288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645597c5-676e-4c4e-bc0e-a4c9d9872b0b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.522831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.523385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.523659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.523921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.524570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:26.524863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.526223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae541bd2-4fff-4d25-b164-9c8aa490c2bf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.534727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.535281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 421/851] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:26.536581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ffb-ed0c-4ccb-8f54-4d5834ca3c98 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.536871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ffb-ed0c-4ccb-8f54-4d5834ca3c98 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.537269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ffb-ed0c-4ccb-8f54-4d5834ca3c98 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.538704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 422/852] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:26.544650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.544919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.545184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.547979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.548601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.548883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.549159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.549823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:26.550104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.551818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.563726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:26.564946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:26.570881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 431/853] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:26.570881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.647077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 423/854] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:26.647077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.670249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.678575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395046d-e9e8-4ad6-a931-ee876ce46cff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.717004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.717670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.717903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.718179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.728922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.730273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:26.746514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:26.747488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:26.753353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:26.753353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:26.753353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:26.753353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 432/855] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:26.755897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.790480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cd58c-0949-42f8-ade8-a84a38a7f1ed None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.800783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.836711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.838233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.838747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.839234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.849138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.850840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.863697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7960c5a-4012-4c98-93c1-96ceddc227ba None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:26.867661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:26.868548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:26.869458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:26.872600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 424/856] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:26.878374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.900404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:26.901495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:26.901778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:26.902053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:26.902828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:26.903301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:26.905040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:26.923077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 433/857] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:26.923077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:26.961690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:26.994016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19b094-e465-46e7-92d9-1a0fc49bb9db None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.029810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.030416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.030693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.031041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.031680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:27.031984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.033096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8668e2-82a3-4a19-accb-a7ba7c20e38f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.033416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.035088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.048467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 425/858] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:27.049210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.080152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.081725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.082473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.082945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.083798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.086507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:27.097999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:27.098961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:27.099888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 434/859] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 31 03:54:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:27.100672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.157340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ab6af-46f6-45ff-9b35-a1c8d2fa9d7d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.187508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.192954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.193626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.193887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.194157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.201864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.203357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.213532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00d0b8-a157-4a22-89d6-260d9c9b5fd0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:27.216070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:27.217107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:27.218044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 426/860] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:27.219015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.253626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.254301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.254606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.254957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.255742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:27.256078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.257499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.264015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 435/861] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/role_assignments?user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:27.275444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.321823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.355627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7fa8-a5f0-4ca9-98bd-8de80f24247c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.403729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.404567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.404942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.405945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.406700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:27.407418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.409269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.416587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 427/862] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:27.422424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.422625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0fc26-3f15-453b-bb89-4ca0819a1188 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.457788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.458513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.458865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.459235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.460042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:27.460454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.461972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.481058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 436/863] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:27.481886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.486779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.538660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf5180-10f0-4303-8977-d11230ee20af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.583329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.584243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.584686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.587431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.588195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:27.588683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.590685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.598492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 428/864] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:27.604643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.623521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83778fa-60a1-48b1-8a07-571fc55bcb17 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.662440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:27.663563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:27.664095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:27.664557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:27.741498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:27.743348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:27.834004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:27.851049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 437/865] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 03:54:27.859058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:27.967818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e25e52-97d9-4120-8c66-d0c8b5a68a2b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:27.989115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:28.005657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.006285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.006556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.006822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.018375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.019852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-511270eb-8824-4901-a18f-de366fb28c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:28.034611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:28.035625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:28.036872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:28.036872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:28.036872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:28.036872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 429/866] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:28.041398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:28.136775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889162d8-21e2-4748-b3ce-431caae61931 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.137277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889162d8-21e2-4748-b3ce-431caae61931 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.137277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889162d8-21e2-4748-b3ce-431caae61931 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.139052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 430/867] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:28.144041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.144382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.144720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.152344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655f3e-3e2d-4f62-bdcc-f9d418aff6a1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:28.163318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:28.164605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:28.191176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.191841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.192114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.192380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.193066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:28.193363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.194771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.207648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 438/868] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:28.216068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:28.239077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 431/869] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:28.272245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:28.329436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84090-1a1d-4af8-ba4b-769ea189e82d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:28.363053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.363922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.364337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.364769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.372237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.373775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:28.385092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:28.385976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:28.386881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 439/870] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:28.392067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:28.495546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ab4a-3779-4fd8-8db8-d00f7b946381 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:28.498132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:28.537107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.537943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.538365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.538644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.539382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:28.539690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.541148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.551738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:28.556070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 432/871] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:28.562271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:28.676641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6211f1-89c0-41d2-850a-2eab6de3d62a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:28.680902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f95fd-8781-430f-b751-c2ef1f801d73 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:28.690108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:28.713615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.714216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.714489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.714753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.715522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=b526cee1dc234083a7f4897fee8e22b6) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:28.715825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.717502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea228-1af3-447d-99c1-959f0ddf9268 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.722558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:28.723336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:28.723670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:28.723944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:28.724177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 433/872] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:28 2022] GET /identity/v3/role_assignments?user.id=b526cee1dc234083a7f4897fee8e22b6&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:28.724610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:28.726002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66694c1-eb42-47e2-bd20-ad4014019ae5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:28.754301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 440/873] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:28 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:29.321115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13485-e073-4593-8a3b-1119fc643465 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:29.322080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13485-e073-4593-8a3b-1119fc643465 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:29.322540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f13485-e073-4593-8a3b-1119fc643465 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:29.324397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 434/874] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:29.330131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486359d-6d3b-4608-b80d-9e187c3fd693 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:29.331013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486359d-6d3b-4608-b80d-9e187c3fd693 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:29.331514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486359d-6d3b-4608-b80d-9e187c3fd693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:29.351203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0486359d-6d3b-4608-b80d-9e187c3fd693 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:29.352690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0486359d-6d3b-4608-b80d-9e187c3fd693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:29.426487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 441/875] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:29.639530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd675fb6-a929-4d45-812c-490a9377728c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:29.639705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd675fb6-a929-4d45-812c-490a9377728c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:29.639982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd675fb6-a929-4d45-812c-490a9377728c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:29.641676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 435/876] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:29.646989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:29.647310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:29.647581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:29.666239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:29.667457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:29.697770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd675fb6-a929-4d45-812c-490a9377728c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:29.727728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:29.747473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 442/877] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:29.811619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd675fb6-a929-4d45-812c-490a9377728c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:29.847396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:29.848393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:29.848860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:29.849352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:29.860647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:29.865755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:29.866705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:29.867577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:29.870378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 436/878] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:29.877143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:29.904221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:29.987535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3e0c2-7f4b-4124-90b2-0bbf774fec46 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.009093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.023840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.024926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.025344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.025752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.027229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:30.027674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.029751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.040312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 443/879] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:30.047148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.107336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:30.125617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cf333-1c21-4351-9d16-ae10fb017099 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.131837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:30.148150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bfc46-e266-4e6b-9c41-8ad4fe512cbf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.166010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.166917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.167409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.167810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.168711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.170556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.183438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.184181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.184451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.184725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.191420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 437/880] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:30 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:30.204236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.206187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:30.223229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:30.224386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:30.225291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:30.225291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:30.225291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:30.227777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 444/881] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:30.234556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.358244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63986901-e928-441e-852f-f6d92d1c46e8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.393246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.393852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.394127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.394393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.395047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:30.395348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.396726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.409507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 438/882] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:30.415977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.520383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e480-b6ed-4b0c-9332-3d1826bad18d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.557277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.557916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.558195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.558466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.561499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:30.568942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.570739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6b5c945a-d125-4322-bc0a-7c5a8989d5c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:30.583443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:30.584451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:30.585413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:30.588135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 445/883] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:30.595059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.597164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:30.705731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d24745-2ecb-4be6-bde6-25c586d50235 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.712024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c3d-dc13-4cdc-a938-d9212f0674da None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.712312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c3d-dc13-4cdc-a938-d9212f0674da None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.712587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c3d-dc13-4cdc-a938-d9212f0674da None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.714184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 446/884] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:30.719454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0c40-ae4b-4c00-94fb-93df499e03d1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.719735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0c40-ae4b-4c00-94fb-93df499e03d1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.720011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0c40-ae4b-4c00-94fb-93df499e03d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.746928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-54ea0c40-ae4b-4c00-94fb-93df499e03d1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:30.746928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-54ea0c40-ae4b-4c00-94fb-93df499e03d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:30.751474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.752263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.752678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.753100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.753972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:30.754411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:30.756548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e07566-1fe3-478e-8be4-092a4a3e1fa6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:30.780336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 439/885] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:30.781744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8b92-a8c2-4dac-bb29-31da7c2bb1c6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.782027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8b92-a8c2-4dac-bb29-31da7c2bb1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.782299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf8b92-a8c2-4dac-bb29-31da7c2bb1c6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.787146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 440/886] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:30.787146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf8b92-a8c2-4dac-bb29-31da7c2bb1c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:30.818075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 447/887] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:30.819539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.819651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.819896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:30.845007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:30.846426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:30.864111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:30.917598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf8b92-a8c2-4dac-bb29-31da7c2bb1c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:30.927836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 448/888] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:30.960479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:30.961374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:30.961827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:30.962235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.034123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.035819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.095651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.107636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.147715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 441/889] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:30 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:31.153058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.175047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.225812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe51b79-681a-4070-9dd1-19319774e733 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.263247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256da70-2f96-438c-a123-b54ae69cdfc2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.268191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.268832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.269109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.269379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.269966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.271803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:31.282018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:31.283051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.283886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 449/890] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:31.284734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.304609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.305253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.305536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.305807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.317723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.319220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:31.335250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:31.336214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:31.337140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:31.337140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:31.337140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:31.339652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 442/891] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:31.340958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.357555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.403623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8edc1-eb07-4977-b1d0-6c12f2d27e37 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.425452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.439923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.440551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.440829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.441099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.441680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.443074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:31.452191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:31.453159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:31.453995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:31.454729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e094-681e-43d2-812d-80d7d8626ecb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.455154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 450/892] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:31.455154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.490364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.490968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.491276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.491531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.492152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.492447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.493840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.499273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 443/893] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:31.500056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.567557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd3d26-6e5a-4d1f-9970-feaebc54a8bd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.579674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.603226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.603861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.604141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.604413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.604649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a611f-2256-46a1-8ce5-f2c5721758ff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.605056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.605356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.606724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.623857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 451/894] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:31.623857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.647253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.647865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.648133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.648403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.649129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.649436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.650808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.659077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 444/895] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:31.659077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.675064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.733493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f444d-f44b-4ece-a793-d04988416bda None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.767078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cca2b-12b6-4b47-b8f5-f4c5d3c30843 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.769502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.770186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.770511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.770780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.771432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.771731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.773075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.778194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 452/896] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:31.779038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.808284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.808919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.809199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.809469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.817359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.818865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.822098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:31.832843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:31.833752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 445/897] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:31.840161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.888993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75493c-caac-4d12-88fb-93d5b7a36111 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.899257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:31.925511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.926106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.926383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.926645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.927290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.927588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:31.928938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:31.943396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 453/898] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:31.944243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:31.945975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5cf72-204a-4833-9a8c-59e7ab6d29b1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:31.996773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:31.997609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:31.997945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:31.998340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:31.999093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] RBAC: Adding query filter params (interface=public, service_id=38a7e1943239497eb38dcb1e9bae8812, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:31.999408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.000847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.009989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 446/899] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/endpoints?service_id=38a7e1943239497eb38dcb1e9bae8812&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:32.015830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:32.076246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d070cf-c6d1-40e1-820c-20d0cd73b424 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:32.106282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:32.123892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:32.124770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.125048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.125317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.126027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:32.126324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.127791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.142973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 454/900] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:32.149091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:32.156580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334b25-32bd-4c46-9950-24272246ab29 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:32.187185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:32.202083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:32.203139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.203688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.204010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.204605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.206751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fa80e-b588-4d8a-8e60-9d18ba178415 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.215533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 447/901] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:32 2022] GET /identity/v3/services => generated 1268 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 03:54:32.216478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e972-63b8-475c-8bbf-cf20e94cfdfd None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.217134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e972-63b8-475c-8bbf-cf20e94cfdfd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.217408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61e972-63b8-475c-8bbf-cf20e94cfdfd None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.219300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 448/902] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:32.228957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.229272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.229510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.255728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:32.257124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:32.278584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237dc8d-9cda-44db-925b-4bd8656d1a24 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:32.329016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:32.330075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.330839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.331311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.332230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] RBAC: Adding query filter params (role.id=28b5e44ef9574928be80ae944f02dd05, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:32.332713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.334933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.343608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 455/903] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:32 2022] GET /identity/v3/role_assignments?user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=28b5e44ef9574928be80ae944f02dd05 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:32.359250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 449/904] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:32.624258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:32.725514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:32.731533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce52ee8-15bc-49c7-b554-877dd1dc46be None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:32.765979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:32.766614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.766886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.767188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.767768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.769122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:32.778188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:32.779216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:32.780095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 456/905] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 31 03:54:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:32.787585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:32.875498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:32.899614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec02b0-f5da-4fdd-8075-7ff745e4b903 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:32.934763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:32.935676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:32.936021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:32.936296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:32.936927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:32.937230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:32.938602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:32.943672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 450/906] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:32.949237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:33.053229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41f29c-be1f-4e0b-8794-6f1956b30276 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:33.089599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:33.090579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:33.091149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:33.091677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:33.092654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:33.093166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:33.094814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:33.100488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 457/907] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:33.106619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:33.214407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf4b7-b4ae-46ae-b101-4d1d65ade067 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:33.238231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:33.250773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:33.251404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:33.251675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:33.251939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:33.252638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] RBAC: Adding query filter params (interface=public, service_id=04c45c47be3b4ff7b1383b029cedf7e3, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:33.252935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:33.254293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5275c-472d-4bd5-9dfa-90b48e4c9b6a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:33.259785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 451/908] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:33 2022] GET /identity/v3/endpoints?service_id=04c45c47be3b4ff7b1383b029cedf7e3&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:33.270722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:33.303012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:33.384725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a12c-d5f4-453f-b586-007acf1b5d29 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:33.420515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:33.421457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:33.421936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:33.422405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:33.423270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:33.424853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f7611-fd44-4dc0-ace5-9980d7e98a21 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:33.431412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 458/909] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:33 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:54:34.026165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5cd98-773c-4549-9e8f-8c9879f633a5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.026700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5cd98-773c-4549-9e8f-8c9879f633a5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.026700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5cd98-773c-4549-9e8f-8c9879f633a5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.028342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 452/910] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:34.033966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abab7-958c-41da-89e2-7a082a18b8a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.034831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abab7-958c-41da-89e2-7a082a18b8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.035332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abab7-958c-41da-89e2-7a082a18b8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.054314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-333abab7-958c-41da-89e2-7a082a18b8a1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:34.055911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-333abab7-958c-41da-89e2-7a082a18b8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:34.124693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 459/911] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:34.153148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a149ac-8145-46be-9fa4-9212259dbb88 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.153853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a149ac-8145-46be-9fa4-9212259dbb88 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.154324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a149ac-8145-46be-9fa4-9212259dbb88 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.156202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 453/912] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:34.161558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b3d-d971-44f1-873c-ef75a4965343 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.162061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b3d-d971-44f1-873c-ef75a4965343 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.162528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b3d-d971-44f1-873c-ef75a4965343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.181484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e4108b3d-d971-44f1-873c-ef75a4965343 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:34.182994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e4108b3d-d971-44f1-873c-ef75a4965343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:34.196734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:34.259144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 460/913] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:34.272277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816423-4976-422c-975c-ee15daf323d5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.272881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816423-4976-422c-975c-ee15daf323d5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.273343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41816423-4976-422c-975c-ee15daf323d5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.275158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 454/914] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:34.280458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.280963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.281428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.301834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:34.303356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:34.371459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 461/915] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:34.401767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41816423-4976-422c-975c-ee15daf323d5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.509521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41816423-4976-422c-975c-ee15daf323d5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:34.529166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.545080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:34.545911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.546329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.546734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.547686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:34.549306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:34.571754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 455/916] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:34 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:34.634366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:34.660945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4772fcd-0d37-4145-9909-5c305ea0dd2e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:34.662057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.716378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:34.718076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:34.719374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.719663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.719934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.720518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:34.721884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:34.731338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:34.732547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:34.733381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 462/917] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:34.738853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.789739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fccb-5b82-4d6a-8684-d126e0fc64c4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:34.836023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:34.837080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.837641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.838145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.849828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:34.850959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d22e8-8d3f-4af9-b926-353a0efe6a82 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:34.855971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:34.856920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:34.857806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:34.860651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 456/918] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:34.866270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.893163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:34.893767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:34.894038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:34.894303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:34.894931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:34.895377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:34.896774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:34.902634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 463/919] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:34.908015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:34.912080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:34.987132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce83968-c4b6-49dc-8aef-1d602fca87bc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:34.994434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.025311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.026033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.026383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.027928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.028693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:35.028993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.030369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.033095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bc7e8-7f2e-4034-8a16-ca9201f39e65 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.043376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 457/920] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:35.045336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.071014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.072323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.072904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.073347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.074245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:35.074743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.076397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.082635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 464/921] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 31 03:54:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:35.087908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.151861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696f8c-5ea7-46ed-b72a-e0c6e173f403 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.160301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.194160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.194798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.195179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.195467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.200012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6490-f40b-4077-b0ed-75b3b1792172 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.207239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.208769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:35.223837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:35.224828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:35.225698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:35.225698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:35.225698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:35.229741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 458/922] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:35.230646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.241109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.242227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.242766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.243271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.244263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.246234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d628-3c05-4a07-8b2f-c5989746f629 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.254215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.274246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 465/923] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:35.364159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e47f6-d9bc-4913-9b76-7784c5b9deda None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.389094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3fd0-2642-4a37-9592-4aea915dadf8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.389443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3fd0-2642-4a37-9592-4aea915dadf8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.394613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e3fd0-2642-4a37-9592-4aea915dadf8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.394613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 466/924] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:35.400856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.401528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.401997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.411931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.412598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.412881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.414074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.414074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:35.414074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.415477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.424870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:35.427100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:35.429147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 459/925] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:35.434666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.527051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 467/926] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:35.527272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.550241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfa970-bad1-4fbf-a7bb-ebc290758c7f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.586086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.586724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.587004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.587309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.595169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.596668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:35.609307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:35.610269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:35.611843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 460/927] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:35.616720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.674159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.724637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878415-2a0a-477c-9f9a-a49ead0518f0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.762834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.763515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.763798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.764190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.764939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:35.765244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.766692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.785744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 468/928] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:35.792114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.804371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.913736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5463-a56b-4f0d-9b92-f36a093b3d2e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.917952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903f37-079c-411f-ae6f-5be35a6e217c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:35.928648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.954472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.955730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.956298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.956787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.957969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:35.958604 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:35.958884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:35.959189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:35.959776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.960862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:35.961310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.971369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:35.972194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:35.973092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:35.979007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 469/929] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:35.979822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:35.981370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:35.993660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:35.994652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:35.996288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:35.996288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:35.996288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:35.998403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 461/930] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:36.009754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.105778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d19bd-2578-4d69-96b5-9eceec70d7e9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.116379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093117e-9e8e-463c-b156-dbba57aac64d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.152117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.153142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.153570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.154185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.154443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.154980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.155398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.155594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:36.155859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.156197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.156686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:36.157096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.158006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.158658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.164519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 470/931] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:36.170087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.185072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 462/932] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:36.186017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.225749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:36.245534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:36.308807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bb084-d44b-4b19-8a58-19e13abae84a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.316787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63881460-6bf5-4629-9bf8-b081e6b726f2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.344936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.345585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.345921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.346191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.347121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:36.347423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.348968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.357682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.358712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.359221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.359693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.360196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 471/933] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:36.361492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.367999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.369716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:36.383224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:36.384157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:36.385105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:36.392418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 463/934] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:36.393548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.491064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c53ec2-f4a4-41e9-8421-d662838f3982 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.492648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:36.517740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3c791-a3e1-446f-81cd-e9e035943865 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.528794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:36.539930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.540568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.540846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.541118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.541786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.543174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de4ffd-e780-4daa-b54c-28bdfa621cb6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.561303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.562543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.563262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.563733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.564753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:36.565364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.567163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.571502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 472/935] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:36.572427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a416f4-14c3-43cb-be5d-7209e8ed6d55 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.572767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a416f4-14c3-43cb-be5d-7209e8ed6d55 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.573041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a416f4-14c3-43cb-be5d-7209e8ed6d55 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.574586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 473/936] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:36.580406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.580724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.580940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.590677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 464/937] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:36.592636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:36.601230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:36.602610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:36.703233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 474/938] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:36.726299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19533-bd2b-4935-841c-6c2818c6b7c3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:36.776171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:36.777182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:36.777776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:36.778247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/5eeb4f0ca36d40e4ad8d5b77cdf35068/roles/28b5e44ef9574928be80ae944f02dd05` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:36.781379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=5eeb4f0ca36d40e4ad8d5b77cdf35068, role_id=28b5e44ef9574928be80ae944f02dd05)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:36.783957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:36.803160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 465/939] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:36 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/5eeb4f0ca36d40e4ad8d5b77cdf35068/roles/28b5e44ef9574928be80ae944f02dd05 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:36.977041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.061153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:37.077272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25756a6-22a1-4af8-aef4-ab348422258c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.120237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.122118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.122938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.123551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.124704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.126307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-11a0c8ba-4fe1-461f-a1fb-9a49a5d9fd75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:37.136058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:37.137088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:37.137976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.140775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 475/940] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:37.147080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.185151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:37.262114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3b086-ac7c-4d5d-a29a-4f35b87bb3df None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.270396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210f8f-a3b4-4954-8460-ab996df6e678 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.270690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210f8f-a3b4-4954-8460-ab996df6e678 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.270953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210f8f-a3b4-4954-8460-ab996df6e678 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.272601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 476/941] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:37.277654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.277940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.278213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.304302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:37.305791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:37.313136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.314057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.314664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.314946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.315618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:37.315922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.317414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.323204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 466/942] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:37.328795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.403231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 477/943] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:37.430153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:37.439901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3d810-7ece-41ed-a61b-db5fcf7e13e8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.474233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.474829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.475133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.475398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.476017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:37.476312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.477732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.483616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 467/944] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:37.489523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.569735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:37.592183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741c918-ce3b-4d71-a4dc-33304acb38b4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.625826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.626420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.626733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.626998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.627721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] RBAC: Adding query filter params (interface=public, service_id=6339cd810b9e42c09945d355ed12e566, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:37.628020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.629388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.634866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 478/945] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:37 2022] GET /identity/v3/endpoints?service_id=6339cd810b9e42c09945d355ed12e566&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:37.640460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.701164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.745148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05029b12-9984-4ced-809b-44c12c3dc758 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.782901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.783554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.783875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.784144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.784751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.786120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.792817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 468/946] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:37 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:54:37.816920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69461-8d48-4b6b-8735-88cd7945db90 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:37.857307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:37.858283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:37.858787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:37.859305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:37.860184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:37.861905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:37.864855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING keystone.server.flask.common [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 03:54:37.880681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:37.887647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 03:54:37.887647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 03:54:37.887647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [parameters: {'id': 'c223862245984dafaf823287219c8342', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 03:54:37.887647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c223862245984dafaf823287219c8342.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c223862245984dafaf823287219c8342. Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:54:37.896379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 03:54:37.897255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Aug 31 03:54:37.898746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 03:54:37.899696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 03:54:37.900629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:54:37.902398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'id': 'c223862245984dafaf823287219c8342', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.904793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:37.906622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c223862245984dafaf823287219c8342. Aug 31 03:54:37.908939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:37.912055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 479/947] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 31 03:54:37 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:37.923085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:37.953538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:38.036053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83261-9a42-4e8c-b94e-aa317655e098 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:38.070721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:38.071383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.071657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.071922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.078737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:38.080334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a3af544e-681b-47ef-b8be-0414bf6d52c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:38.091746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:38.092855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:38.093942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 469/948] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:38.100808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:38.218375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44108f1-a31d-4081-b214-cc3da81a088a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:38.261761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:38.262380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.262661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.262995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.263836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:38.269510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:38.271287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aac75-805b-45bc-b394-07f9cd539d11 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:38.298902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 480/949] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:38.662025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765afab-b047-4e99-82c8-e32965c64462 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.662565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765afab-b047-4e99-82c8-e32965c64462 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.662649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765afab-b047-4e99-82c8-e32965c64462 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.664346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 470/950] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:38.670030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.670351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.670606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.689950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:38.691502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:38.749843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:38.760210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5907-b0d0-4be2-8e61-30db822235f8 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.760657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5907-b0d0-4be2-8e61-30db822235f8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.760924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5907-b0d0-4be2-8e61-30db822235f8 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.763271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 471/951] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:38.767608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 481/952] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:38.771305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:38.771585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:38.771855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:38.790496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:38.791714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:38.860932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 472/953] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:39.036769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.139735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.142250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e19d28-04ba-4c06-b30f-af5c2a27854b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.151822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.179735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.180367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.180647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.180921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.181510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.182879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:39.192433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:39.193429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:39.194479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 482/954] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:39.201467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.251994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e161c7-247b-46df-a88f-f32b7055067b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.260067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.289352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.290254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.290733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.291228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.307603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:39.313127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:39.314128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:39.315135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:39.317809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 473/955] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:39.323653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.328979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd389b5-8b70-4882-b220-d04f14bebea8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.365185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.365795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.366077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.366352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.367130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:39.367439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.368813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.379194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 483/956] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:39.380816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.399663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.433549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bdd27-e98d-46df-929f-728a9280fb1a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.471513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.472527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.473006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.473474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.474378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:39.474875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.476464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.485043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 474/957] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:39.491486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.501164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59550ca9-14aa-4c53-93f2-e512f26adf05 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.521453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.545237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.546556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.547058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.547580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.548775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:39.549575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.551452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.558080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 484/958] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:39.617872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91221-4bda-4b06-9453-b94b2f589360 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.651920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.658437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.659102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.659386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.659665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.672965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.674469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fe123ed9-8d92-4bbe-99b1-15a60252f166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:39.690319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:39.691421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:39.692361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:39.692361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:39.692361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:39.692361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 475/959] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:39.698166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.777002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98c10-c97e-4380-83a0-e711085592ea None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.777378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98c10-c97e-4380-83a0-e711085592ea None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.777550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98c10-c97e-4380-83a0-e711085592ea None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.779431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 476/960] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:39.787607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.787959 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.788299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.807543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:39.808755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:39.811800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e4fe5-36c9-4835-b397-a60ff79e5e8c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:39.848099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:39.849063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:39.849525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:39.849981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:39.850817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:39.851323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:39.852980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:39.866443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 485/961] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:39.873088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:39.885702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 477/962] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:39.922648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:39.990093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bea77-6727-46bb-a58c-f2ce201cf1be None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.026172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.026848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.027175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.027451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.036688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.038145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:40.050835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:40.051875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:40.053452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 486/963] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:54:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:40.058223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.153111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.169555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635c38e-c705-4a7d-8343-a153faa3a3ac None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.190894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.223849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.225427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.225877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.226295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.227215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:40.227955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.230437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.256739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 478/964] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:40.257772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.279987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.289561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816ec24-dc5c-4570-a2b8-37055b5b95f6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.326719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.327610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.328043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.328318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.328906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.330278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b11659b5-2f8b-4737-abf2-1feaee337ded None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:40.339097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:40.345327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:40.346390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 487/965] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:40.347807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9736a-2504-472d-9c55-ca826306a4a1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.347807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9736a-2504-472d-9c55-ca826306a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.347807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9736a-2504-472d-9c55-ca826306a4a1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.347807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 488/966] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:40.348350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9736a-2504-472d-9c55-ca826306a4a1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.377620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f2672-38f8-4e0c-8111-549545fb04d2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.411438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.412352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.412822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.413092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.456176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.457025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9736a-2504-472d-9c55-ca826306a4a1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.483733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.485185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.493342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.493766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.494049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.494351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.494978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:40.495307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.496978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69967eea-d668-4565-823f-455d2fc50da2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.502367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 489/967] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:40.503375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.503660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.503930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.521870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:40.523080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:40.547159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.595210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 479/968] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:40.595210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.596764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 490/969] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:40.598496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.702418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327efe-abec-452d-bda3-f749a694ec2c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.703267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.740293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.741150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.741440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.741811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.745987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a3dc3-c1e3-4eae-9f6d-1088f44b5023 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.753090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.754669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:40.769797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:40.770933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:40.772119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:40.772119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:40.772119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:40.778035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 491/970] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:40.780076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.793315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.794096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.794474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.794888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.795768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:40.796163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.797101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:40.798970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.806082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 480/971] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:40.811367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.899815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3715ce-c50b-4356-a712-7ae3d181afa2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.929671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84001eb-f6b0-413c-b0a1-8becae6bff6f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:40.932886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.933486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.933760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.934026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.934727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:40.935111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.936473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.947970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 492/972] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:40.948735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:40.968944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:40.969562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:40.969820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:40.970087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:40.970757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:40.972139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:40.978073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:41.003411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 481/973] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:41.004321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:41.035520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:41.073826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a6b66-70eb-4514-bd06-13157044befe None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.118784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.119426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.119758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.120032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.120996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.122627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.138237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 03:54:41.138237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 03:54:41.138237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [parameters: {'user_id': '1ecdb3bc8d024c4b88f60c4368267712', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 03:54:41.138237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 03:54:41.141370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7536f-539d-4b7e-bb25-6945743a6e33 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:54:41.145611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:54:41.146708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 03:54:41.147828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 03:54:41.148910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 03:54:41.149960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 03:54:41.150876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'user_id': '1ecdb3bc8d024c4b88f60c4368267712', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:41.151959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 03:54:41.152824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 493/974] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 31 03:54:40 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:41.154039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:41.189339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.189975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.190250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.190520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.197601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.199079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:41.211393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:41.212423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:41.213346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:41.215281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 482/975] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:41.221436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:41.283396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd909-5eae-4a0d-bd8f-b9290caab2f2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.308868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:41.324490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:41.337092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.337876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.338043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.338364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.346787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.348232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:41.360460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:41.361674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:41.362828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 494/976] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:41.375201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:41.375373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527af7a6-73f8-441c-bd4e-d239b93b7310 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.415582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.416161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.416576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.416920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.417680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:41.418090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.419795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.435269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 483/977] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:41.438845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:41.509958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7384-7f58-4f36-94f1-fcd146246e63 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.548077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116417b-9795-4ac0-a5eb-76c1f0edaffe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:41.558941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.559586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.559992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.560269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.560983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:41.561293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.562726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.564390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:41.585912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:41.586515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.586791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.587080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.587811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] RBAC: Adding query filter params (role.id=28b5e44ef9574928be80ae944f02dd05, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:41.588120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:41.589558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f2da-b489-4ec8-82ca-1973e9e1555e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:41.595421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 484/978] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:41 2022] GET /identity/v3/role_assignments?user.id=5eeb4f0ca36d40e4ad8d5b77cdf35068&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=28b5e44ef9574928be80ae944f02dd05 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:41.596653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a94f8-9a6d-4235-836e-6c81e244ba78 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.596929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a94f8-9a6d-4235-836e-6c81e244ba78 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.597198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a94f8-9a6d-4235-836e-6c81e244ba78 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.598384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 485/979] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:41.604225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33e9c-e6db-43f0-ac29-cfa9b1af8719 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:41.604506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33e9c-e6db-43f0-ac29-cfa9b1af8719 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:41.604781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b33e9c-e6db-43f0-ac29-cfa9b1af8719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:41.611589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 495/980] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 31 03:54:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:41.630798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a9b33e9c-e6db-43f0-ac29-cfa9b1af8719 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:41.632450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a9b33e9c-e6db-43f0-ac29-cfa9b1af8719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:41.716983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 486/981] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:41.980942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:42.033203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:42.089395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421fdc-3a15-48c9-aa13-7076f0a8b257 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:42.125012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:42.125618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:42.125894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:42.126164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:42.126828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:42.128245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec151290-2f20-414b-b556-b26447b40563 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:42.145481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 496/982] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:41 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:42.991587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f51389-2a76-421e-8c88-23345c3fe293 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:42.992137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f51389-2a76-421e-8c88-23345c3fe293 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:42.992137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f51389-2a76-421e-8c88-23345c3fe293 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:42.993613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 487/983] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:42.999191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:42.999623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:42.999726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.018272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:43.019797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:43.097299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 497/984] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:43.359915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f51389-2a76-421e-8c88-23345c3fe293 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:43.468501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f51389-2a76-421e-8c88-23345c3fe293 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:43.476243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:43.504389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:43.505023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.505301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.505569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.506149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:43.507562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bfd5429e-fb74-4bf3-b36f-d3006bd63ffc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:43.517239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:43.518228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:43.520056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 488/985] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:43.524586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:43.556569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:43.565389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a22f44-703f-4865-bca9-d8e997259936 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.566167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a22f44-703f-4865-bca9-d8e997259936 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.566594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a22f44-703f-4865-bca9-d8e997259936 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.568707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 489/986] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:43.574050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.574497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.575100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.596271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:43.597840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:43.636406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed31d1c-5d34-4b01-a873-942f1e6db8b1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:43.675085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 490/987] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:43.676098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:43.676718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.676996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.677261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.677906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:43.678207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:43.679605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:43.689928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 498/988] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:43.691175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:43.794856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:43.798466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c854f-77bf-4e30-9358-6cdf036b17b5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:43.834616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:43.835239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.835517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.835786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.836423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:43.836718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:43.838094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:43.848185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 491/989] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:43.849161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:43.936370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:43.947088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:43.953761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926cc39-b449-4513-8439-79e7fbc14cba None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:43.988529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:43.989286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:43.989565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:43.989834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:43.990545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] RBAC: Adding query filter params (interface=admin, service_id=6339cd810b9e42c09945d355ed12e566, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:43.990847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:43.992264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:43.997884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 499/990] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Aug 31 03:54:43 2022] GET /identity/v3/endpoints?service_id=6339cd810b9e42c09945d355ed12e566&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:44.003584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.050433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46add247-1aac-40fb-8693-a51b2262fc68 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.084194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.084887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.085098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.085358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] PATH_INFO: `/v3/services/image` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.085923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.087327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f0bf045a-3e63-4845-9561-9feb80b10323 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:44.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:44.097438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.098257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 492/991] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 31 03:54:43 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:44.099634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1dfa4-c90f-43f7-9a5b-5d5c5b8dbed7 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.099634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1dfa4-c90f-43f7-9a5b-5d5c5b8dbed7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.099634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1dfa4-c90f-43f7-9a5b-5d5c5b8dbed7 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.103060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 493/992] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:44.103355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1dfa4-c90f-43f7-9a5b-5d5c5b8dbed7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.109039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b023ee-1525-4abe-8d99-b563b22c7c7b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.131873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:44.144794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.145396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.145666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.145934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.146527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.147915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b7d47-ea55-4952-8fe7-4b31fd6e7a22 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.154120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 500/993] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:54:44.155299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.155578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.155846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.180642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:44.181868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:44.203293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:44.216579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1dfa4-c90f-43f7-9a5b-5d5c5b8dbed7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.256122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.256733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.257018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.257285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.257928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] RBAC: Adding query filter params (name=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:44.258219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.259670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.271065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 501/994] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:44.271286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 494/995] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:44.271386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.377282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ec7dd-873b-4635-b686-34495c5d9444 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.410945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.411600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.411849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.412127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.412757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] RBAC: Adding query filter params (type=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:44.413050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.414427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.419464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 502/996] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:44.537646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.643381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:44.652752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7f25-5471-4fd7-ad9b-a0a78a0abf32 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.688992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.689626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.689902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.690169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.690752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.692153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:44.702755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:44.703943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:44.704777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:44.708843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 495/997] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:44.709876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.730715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:44.817840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb691ba4-6b9e-4b91-b733-1165376bb514 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:44.854439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:44.855059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:44.855337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:44.855604 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:44.856233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:44.856582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:44.858309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:44.864465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 503/998] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:44.870079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:44.974123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccce-b7aa-453c-89a9-3139be241df4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:45.008914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:45.009519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:45.009794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:45.010057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:45.010685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:45.010982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:45.012454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:45.017989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 496/999] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:45.023797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:45.106007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:45.126468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e80a1a-1d27-47a9-8fde-910fdefd1a6b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:45.164010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:45.164612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:45.164889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:45.165153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:45.165847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] RBAC: Adding query filter params (interface=public, service_id=334f491be26c4979ac25d752661e961d, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:45.166144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:45.167531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efa36f-8f07-4547-a391-4d0471a8df24 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:45.173557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 504/1000] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:45 2022] GET /identity/v3/endpoints?service_id=334f491be26c4979ac25d752661e961d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:45.178902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:45.201806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:45.290377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f20b3-99d0-4b0f-b588-9ed61c4cd98f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:45.326414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:45.327010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:45.327339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:45.327608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:45.328207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:45.329612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0e9bb-1e19-4eb0-bdad-2723c56d665e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:45.336411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 497/1001] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:45 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:54:46.105514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded5428-3e40-4595-9480-f68be69b3eb3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.106102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded5428-3e40-4595-9480-f68be69b3eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.106102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded5428-3e40-4595-9480-f68be69b3eb3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.107741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 505/1002] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:46.112804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a34b5-07e2-4c20-b501-5eda8ccfdfb9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.113072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a34b5-07e2-4c20-b501-5eda8ccfdfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.113362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a34b5-07e2-4c20-b501-5eda8ccfdfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.132682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-473a34b5-07e2-4c20-b501-5eda8ccfdfb9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:46.133865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-473a34b5-07e2-4c20-b501-5eda8ccfdfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:46.204526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 498/1003] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:46.305516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08e5e-2362-4400-be01-7c059352dce6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.305819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08e5e-2362-4400-be01-7c059352dce6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.306092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08e5e-2362-4400-be01-7c059352dce6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.307683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 506/1004] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:46.313115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.313414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.313659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.332256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:46.333433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:46.344141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:46.407683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 499/1005] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:46.470440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08e5e-2362-4400-be01-7c059352dce6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:46.576970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08e5e-2362-4400-be01-7c059352dce6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:46.611577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:46.612600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.613047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.613493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.614417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:46.616638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:46.631538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:46.632533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 03:54:46.633392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:46.634834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 507/1006] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 31 03:54:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:46.641716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:46.672660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:46.686904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:46.751336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2b7bb-7348-4488-8ad2-bca9a4a284b8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:46.787349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:46.787934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.788227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.788489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.789262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:46.789388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:46.790107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc09ac-c03e-436e-89ed-71451fb0b98c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:46.790786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:46.797103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 500/1007] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:46.802720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:46.814669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:46.835772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:46.836384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.836661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.836931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.837591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:46.838963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:46.855570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 508/1008] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:46 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:54:46.913451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac65f3-0d30-412a-a1fb-22b058b2bf31 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:46.959504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:46.960172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:46.960449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:46.960678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:46.961338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:46.961630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:46.963014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f8615-1379-4a3a-8460-d693b1636b3f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:46.968958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 501/1009] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 31 03:54:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:46.974877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:47.050513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:47.083661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681304-ee11-4d5c-90d5-2112f80d2ad2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:47.119370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:47.120070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:47.120306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:47.120814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:47.121478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:47.123533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffe1e1-0565-4889-ba73-3dadfa3dff12 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:47.141131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 509/1010] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:47.232788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95cf1-1ce0-4d70-bce5-53d1a8033999 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:47.233332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95cf1-1ce0-4d70-bce5-53d1a8033999 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:47.233332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95cf1-1ce0-4d70-bce5-53d1a8033999 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:47.234819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 502/1011] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:47.240348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:47.240634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:47.240903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:47.261010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:47.262222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:47.347650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 510/1012] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:47 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:47.611565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95cf1-1ce0-4d70-bce5-53d1a8033999 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:47.683640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:47.719791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95cf1-1ce0-4d70-bce5-53d1a8033999 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:47.755658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:47.756276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:47.756518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:47.756812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:47.757373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:47.758777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:47.767917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:47.769039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:47.769915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 503/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:47.776860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:47.836586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:47.883863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e4c92-8b19-48b6-b062-8cbcd446d114 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:47.920009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:47.920618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:47.920898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:47.921165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:47.921803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:47.922103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:47.923515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:47.928650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 511/1014] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:47.934297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:48.038391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3eed-6dc2-41e5-ab54-7a34df39a77a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:48.067756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:48.074149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:48.074743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:48.075041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:48.075357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:48.075973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:48.076268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:48.077701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8fb21-ea84-4e4f-b265-17cf94d78160 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:48.082861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 504/1015] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 31 03:54:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:48.088535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:48.191455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f9c02-74bb-41e6-a0e0-690575cdfdb4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:48.212850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:48.229798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:48.230404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:48.230673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:48.230938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:48.231705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:48.233063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b019e00-760f-4d50-93ab-7bcee4cecd3c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:48.255503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 512/1016] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:54:48.772695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7b5e-6e61-47ba-9257-d8baf6b373cd None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:48.773231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7b5e-6e61-47ba-9257-d8baf6b373cd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:48.773231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe7b5e-6e61-47ba-9257-d8baf6b373cd None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:48.774614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 505/1017] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:48.779967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89235dc4-7062-45b7-901e-ae77f8a3e612 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:48.780178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89235dc4-7062-45b7-901e-ae77f8a3e612 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:48.780498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89235dc4-7062-45b7-901e-ae77f8a3e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:48.799789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-89235dc4-7062-45b7-901e-ae77f8a3e612 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:48.800939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-89235dc4-7062-45b7-901e-ae77f8a3e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:48.879061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 513/1018] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:49.044394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bc16e-1e11-430a-8284-35b17ca84560 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.044599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bc16e-1e11-430a-8284-35b17ca84560 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.044853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bc16e-1e11-430a-8284-35b17ca84560 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.046377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 506/1019] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:49.051483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.051768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.052007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.070770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:49.071985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:49.140389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 514/1020] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:49.141939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bc16e-1e11-430a-8284-35b17ca84560 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.248492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bc16e-1e11-430a-8284-35b17ca84560 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.288094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.288734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.289009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.289283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.289868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.291261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:49.300294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:49.301271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:49.302109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:49.306214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 507/1021] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:49.307498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.342051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:49.407549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.416998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ba447-e05c-4468-be6c-261b63a3b029 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.452762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.453522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.453635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.453893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.454520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] RBAC: Adding query filter params (name=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:49.454818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.456280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.461497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 515/1022] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:49.467218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.476574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:49.520436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fbaed-8802-4286-8c80-a8ed2e4affc7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.558885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.559601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.559881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.560149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.561257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.562702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.569158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 508/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:49.569652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:49.569933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.576697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c30ca3-ce6b-43d6-8ef4-e6d8f1d1f948 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.612815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.613419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.613670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.613965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.614560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] RBAC: Adding query filter params (type=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:49.614856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.616282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.622323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 516/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:49.628341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.679876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ab6b-4005-49b5-9ace-65cc3eb31021 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.711104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:49.720617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.721213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.721489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.721775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.722433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.723916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.736295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38812af-8a18-44ba-af2b-0522524ee62e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.756326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 509/1025] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:54:49 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:54:49.774890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.775558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.775836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.776105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.776838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] RBAC: Adding query filter params (interface=public, service_id=e35b1d6e8e0e42f3a4bf2dfc61be6b8a, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:49.777146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.778633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf9105-80b7-471b-b50b-15a059b819bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:49.784933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 517/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/endpoints?service_id=e35b1d6e8e0e42f3a4bf2dfc61be6b8a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:49.795177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:49.949713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01556-173f-4942-9a2e-3409165e49e3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:49.988743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:49.989359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:49.989636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:49.989906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:49.990515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:49.991941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f7ac6-fe25-4c57-86d9-ca065b13b89f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:50.002166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 510/1027] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:54:49 2022] GET /identity/v3/services => generated 1957 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:54:50.257724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002d4bf-97d9-4b3d-8b4f-b8100aa844c2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:50.257911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002d4bf-97d9-4b3d-8b4f-b8100aa844c2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:50.258186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002d4bf-97d9-4b3d-8b4f-b8100aa844c2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:50.259707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 518/1028] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:50.267482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:50.267482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:50.267482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:50.286771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:50.288428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:50.343948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:50.374901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 511/1029] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:50.683819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002d4bf-97d9-4b3d-8b4f-b8100aa844c2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:50.716853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:50.829560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002d4bf-97d9-4b3d-8b4f-b8100aa844c2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:50.871557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:50.872237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:50.872517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:50.872786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:50.883247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:50.888248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:50.889155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:50.890004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:50.894436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 519/1030] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:50.895372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.000353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e0671-e99a-4f62-81fd-454dba815f8a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.036260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.036852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.037120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.037387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.038044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:51.038334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.039726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.048560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 512/1031] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:51.054656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.141251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:51.159133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509dd72-6f05-41c0-9c85-bd9f909276a8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.194299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.194924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.195228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.195501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.206500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.207972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:51.223036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:51.224016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:51.224923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:51.224923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:51.224923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:51.224923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 520/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:51.231828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.273279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:51.342081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f8a87-2193-48f8-a26b-9f8366d93542 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.377474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.378071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.378342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.378605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.379282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:51.379573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.380934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.393406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 513/1033] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:51.398832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.498968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b52888-3502-417b-a8c5-a80b9372a6fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.532543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.533080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.533353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.533614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.536355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:51.542627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.544351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:51.555773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:51.556734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:51.557693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:51.559194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 521/1034] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:51.562921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.662429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:51.669002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668809-bab9-4cd3-9b19-75a476cbfdc3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.708213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.708826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.709104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.709372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.710104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:51.710403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.711865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160046b-7ba6-403e-aa05-3f4ac21d075b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.730697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 514/1035] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:51.737184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:51.787940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf142-0b9b-4947-93c8-5104eb7ec05a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.788124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf142-0b9b-4947-93c8-5104eb7ec05a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.788429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf142-0b9b-4947-93c8-5104eb7ec05a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.789908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 515/1036] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:51.794622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dff9f-5057-43be-94ce-41a0b07f4d4a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.794908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dff9f-5057-43be-94ce-41a0b07f4d4a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.795159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dff9f-5057-43be-94ce-41a0b07f4d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.813679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-929dff9f-5057-43be-94ce-41a0b07f4d4a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:51.814861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-929dff9f-5057-43be-94ce-41a0b07f4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:51.848683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb53fe-d104-4aff-ae9f-ba35c6423461 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:51.882946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:51.883584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.883869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.884139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.892464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 516/1037] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:51.950442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:51.957195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:51.958660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:51.977759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6759c-e829-4645-a341-5b6be8a7b7fc None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.978275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6759c-e829-4645-a341-5b6be8a7b7fc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.978275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6759c-e829-4645-a341-5b6be8a7b7fc None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:51.979677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 517/1038] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:51.985197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:51.985480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:51.985754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.005161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:52.006380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:52.074942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 522/1039] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:52.077332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 518/1040] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:52.080445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.151519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.168523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.177764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.185007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259bbd7-c53e-4c29-9fd4-222a00611530 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.220571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.221192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.221469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.221738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.232190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.233626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:52.249110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:52.250268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:52.251559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:52.251559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:52.251559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:52.251559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 523/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:52.255935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.268162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7bb1a-cc2f-494e-aeea-9109046031cb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.303653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.304283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.304557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.304824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.305919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.307378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.309126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 519/1042] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:52.314691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.368165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fad97-3ffc-4987-8d66-536c4ac19317 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.397150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.401122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.401726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.401999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.402265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.402899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:52.403265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.404637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.416119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7c0-71f0-48c7-939e-6972fc93a9ea None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.416448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 524/1043] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:52.421462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.427372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.457272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.457880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.458160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.458455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.459130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.460599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.482831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 520/1044] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:54:52 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 03:54:52.483680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.531413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eab457-09d4-4d6a-a093-b736063e907c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.577447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.578144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.578427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.578701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] PATH_INFO: `/v3/services/image` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.579328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.580743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:52.590381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:52.591827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:52.592677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 525/1045] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:52.597867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.603332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9e54f-74b9-4a3c-833d-b767ef0c6bb7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.646667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.647296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.647662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.647854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.655917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.657371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.660127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:52.671601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:52.672553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:52.673478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 521/1046] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:52.680549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.685776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.724484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda9f30-2ffa-4c26-8e42-1c25eac06e9d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.760410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.761081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.761303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.761550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.762183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] RBAC: Adding query filter params (name=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:52.762510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.763913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.769142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 526/1047] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:52.774525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.798576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a9722-ae07-4eec-86f1-df5a48fd0bf5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.837041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.837795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.838164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.838519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.839343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:52.839739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.841938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.854358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 522/1048] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:52.860613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.885945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed9dd6-70c8-4c52-8806-e44b742e9633 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:52.922104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.924809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:52.925609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:52.925982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:52.926385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:52.927039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] RBAC: Adding query filter params (type=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:52.927344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:52.928771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:52.930967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:52.934755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 527/1049] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:52.943900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:52.970800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea45ce7-588d-4cd7-ba82-22f673909d58 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:53.008336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:53.009021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:53.009365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:53.009700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:53.010489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] RBAC: Adding query filter params (role.id=28b5e44ef9574928be80ae944f02dd05, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:53.010853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:53.012530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765b8b5-de9f-415d-8167-620e56f13ea1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:53.018704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 523/1050] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:52 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=28b5e44ef9574928be80ae944f02dd05 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:53.054093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197387f-491b-4e7a-9f93-4be2cdd4583d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:53.094558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:53.095212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:53.095494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:53.095756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:53.096430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:53.097836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02456f12-33ac-499a-bb76-079372be7693 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:53.131065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 528/1051] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:54:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:54:54.425367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dbbfa-d4cf-46cc-95e0-189a91034cd9 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:54.435254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dbbfa-d4cf-46cc-95e0-189a91034cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:54.435254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24dbbfa-d4cf-46cc-95e0-189a91034cd9 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:54.435254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 524/1052] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:54.437694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c5f8-2c9f-43b9-b012-e483d35a28f9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:54.438178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c5f8-2c9f-43b9-b012-e483d35a28f9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:54.438261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c5f8-2c9f-43b9-b012-e483d35a28f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:54.456945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0e64c5f8-2c9f-43b9-b012-e483d35a28f9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:54.458132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e64c5f8-2c9f-43b9-b012-e483d35a28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:54.509032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:54.533964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 529/1053] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:54.801716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dbbfa-d4cf-46cc-95e0-189a91034cd9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:54.901855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99986-2272-4b15-93bc-bf00d0b69f5a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:54.902211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99986-2272-4b15-93bc-bf00d0b69f5a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:54.902641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99986-2272-4b15-93bc-bf00d0b69f5a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:54.904527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 530/1054] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:54.909711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd9c1-7f7e-4634-baf7-b1f8482ccb77 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:54.910007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd9c1-7f7e-4634-baf7-b1f8482ccb77 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:54.910279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd9c1-7f7e-4634-baf7-b1f8482ccb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:54.915597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24dbbfa-d4cf-46cc-95e0-189a91034cd9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:54.929688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-609dd9c1-7f7e-4634-baf7-b1f8482ccb77 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:54.930882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-609dd9c1-7f7e-4634-baf7-b1f8482ccb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:54.941195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:54.955304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:54.955930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:54.956200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:54.956471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:54.967437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:54.972267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:54.973272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:54.974721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 525/1055] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:54.979228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.009623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 531/1056] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:55.032959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa516a-d071-4a90-8df5-d72eec8d19ce None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.033560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa516a-d071-4a90-8df5-d72eec8d19ce None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.034066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa516a-d071-4a90-8df5-d72eec8d19ce None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.035844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 532/1057] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:54:55.040478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.040757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.041022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.059065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:55.060447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:55.091073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cb4bd-9604-4b57-b0e9-3ed4865faaba None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.126315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.126918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.127227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.127639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 533/1058] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 03:54:55.131281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.132063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:55.132373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.133751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.142426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 526/1059] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:55.148098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.192237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.252172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da8abf-63fd-45a5-8f53-7ef5f8f9793e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.277237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.298818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.299489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.299771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.300045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] PATH_INFO: `/v3/users/647fff81c4cb46e082eb687705426dfd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.318303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] RBAC: Authorizing `identity:get_user(user_id=647fff81c4cb46e082eb687705426dfd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.319718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.326139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 534/1060] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/users/647fff81c4cb46e082eb687705426dfd => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:55.327273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.329604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.388079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ac68-a3f7-45b5-a6dc-b88e1e089826 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.422594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.423252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.423532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.423802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.427791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.430759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e17021-02c7-452a-a88a-3c29f6ef8339 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.434787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:55.439646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:55.440736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:55.441565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 527/1061] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:55.442769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.472434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.473116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.473400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.473671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.543739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.545289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.547336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3ca1d-81f8-46de-920f-186f5a66e7e5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.550878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.588595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.589511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.589979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.590467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.650616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.659195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 535/1062] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:55.661348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.662692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.664309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.768925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 528/1063] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:54:55.769760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.772734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.775549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a364-3f05-4107-a2d2-7ee0fa0e1949 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.808956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.809811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.810037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.810308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.810971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:55.811345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.812713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.820896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 536/1064] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:55.822241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.868452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ea42-3b18-49ca-af3c-b259d4ae88dd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.886014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.902976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.903705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.904001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.904279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.905403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.906839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.908542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 529/1065] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:55.909281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:55.926007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0fec-e306-4e42-8b3a-6903ec10c7c4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:55.959984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:55.960609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:55.960885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:55.961151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:55.971872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:55.973374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:55.978146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:55.987586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:55.988599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:55.989507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:55.989507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:55.989507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:55.989507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 537/1066] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:55.989507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.010869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767efe-5410-49fa-8cef-fb36d6bfac50 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.045665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.046291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.046546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.046824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.056776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.058231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:56.072974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:56.073944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:56.074890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:56.074890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:56.074890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:56.074890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 530/1067] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:56.075173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.096133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47f978-0844-46e8-9805-306c7de0637f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.109920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:56.131180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.131711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.131983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.132247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.132959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6e7cb145a47348e189557322c98b0682, user.id=647fff81c4cb46e082eb687705426dfd) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:56.133258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.134671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.140016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 538/1068] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:54:55 2022] GET /identity/v3/role_assignments?user.id=647fff81c4cb46e082eb687705426dfd&scope.project.id=6e7cb145a47348e189557322c98b0682&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:56.140775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.206094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0282-f5a9-40a9-bd85-b16a40509c6a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.233693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:56.248885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.249510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.249794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.249973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff554-70fc-45a8-b6ea-c61e9f3ff583 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.250082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.250721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:56.251126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.252496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.275069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 531/1069] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:56.302192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.302934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.303308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.303651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.304372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:56.304745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.306226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.324952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 539/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:56.326699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.438557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a40d5f-8816-4422-bb16-13bd17d32cd5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.476989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.477620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.477900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.478171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.486220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.487717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:54:56.500269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:56.501468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:56.502644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 532/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:56.509579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.563981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:56.624423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e9214-46cf-4068-989c-ad2168b43bb0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.658838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.659462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.659753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.660018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.660734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:56.661033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.662445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.683576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 540/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:54:56.689740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.739045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:56.795845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb03d1-6c25-41ec-ac53-a524004b2148 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:56.829696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:56.830302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:56.830576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:56.830844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:56.841800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:56.843277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:54:56.857830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:56.858974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:54:56.860139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:54:56.860139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:54:56.860139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:56.863218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 533/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:56.864807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:56.968133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:56.974922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b15e3-1653-43a2-a2b3-7c578be56d79 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:57.011477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:57.012071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:57.012537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:57.013112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:57.013758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:57.014063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:57.015451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:57.026640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 541/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:57.033139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:57.133322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:57.139733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783c1ee-5a2a-46bb-a6bc-0d0400579b61 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:57.176038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:57.176650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:57.176951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:57.177218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:57.185860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:57.187387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:54:57.200156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:57.201132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:57.202012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 534/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:54:57.208008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:57.317719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ad7d9-17c0-442d-9719-397326102f8f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:57.352500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:57.353091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:57.353369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:57.353646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:57.354329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:57.354633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:57.356267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c219-5bb3-4ac4-afa2-0a830b0d9d36 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:57.369199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 542/1076] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:57.376448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:57.476220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1233efa-8a61-49bd-921a-c51856cc40ff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:57.511535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:57.512213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:57.512564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:57.512934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/28b5e44ef9574928be80ae944f02dd05` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:57.515162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=ed122ea299374f9cb2b8b4854df14d65, role_id=28b5e44ef9574928be80ae944f02dd05)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:57.516632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee91a-2438-4988-9dd9-438a4366846d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:57.531248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 535/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:57 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/ed122ea299374f9cb2b8b4854df14d65/roles/28b5e44ef9574928be80ae944f02dd05 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:54:57.998246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4984015-99b9-4ee2-98ae-c861a236f0b3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:57.998850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4984015-99b9-4ee2-98ae-c861a236f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:57.998850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4984015-99b9-4ee2-98ae-c861a236f0b3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.000343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 543/1078] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:58.005399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:58.005689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:58.005963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.025750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:58.026962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:58.038348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:58.098310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 536/1079] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:58.362900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4984015-99b9-4ee2-98ae-c861a236f0b3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:58.383115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:58.469397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4984015-99b9-4ee2-98ae-c861a236f0b3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:58.504236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:58.504859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:58.505130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:58.505396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.515486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:58.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:58.521166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:54:58.521998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 544/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:58.527438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:58.635571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5d1c2-741c-49ea-ae26-44cdd5bef851 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:58.671646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:58.672242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:58.672515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:58.672778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.673426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:54:58.673721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:58.675082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:58.683268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 537/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:58.689183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:58.770008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:58.795433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ccd6-1a45-4290-b0a5-a4793b881224 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:58.832534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:58.833533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:58.834005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:58.834482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] PATH_INFO: `/v3/users/647fff81c4cb46e082eb687705426dfd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.836532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] RBAC: Authorizing `identity:get_user(user_id=647fff81c4cb46e082eb687705426dfd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:58.839250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:58.841992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 545/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:54:58 2022] GET /identity/v3/users/647fff81c4cb46e082eb687705426dfd => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:58.848212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:58.860204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:58.952275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4757-d999-482e-baf5-1334bfb3779c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:58.987780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:58.988404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:58.988679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:58.988947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:58.990104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:58.991570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc215e9-1799-4b05-aa05-8d82b67540ac None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:58.993219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 538/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:54:58 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:54:59.000066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:59.099070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a1901-4f8e-4403-8e36-5cb128ee69a3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:59.144369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:59.145047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:59.145327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:59.145606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682/users/647fff81c4cb46e082eb687705426dfd/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:59.147682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e7cb145a47348e189557322c98b0682, user_id=647fff81c4cb46e082eb687705426dfd, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:54:59.149135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b627d-f9c7-4edb-b64e-98fdd3252a81 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:54:59.165085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 546/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:54:58 2022] PUT /identity/v3/projects/6e7cb145a47348e189557322c98b0682/users/647fff81c4cb46e082eb687705426dfd/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 03:54:59.413689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dee7be-443a-421b-b0d2-b4d52924d08d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:59.414082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dee7be-443a-421b-b0d2-b4d52924d08d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:59.414358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dee7be-443a-421b-b0d2-b4d52924d08d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:59.415757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 539/1085] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:54:59.421270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:59.421555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:59.421827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:59.429852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:59.441788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:54:59.442985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:54:59.514533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 547/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:54:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:54:59.775124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dee7be-443a-421b-b0d2-b4d52924d08d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:54:59.855702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:54:59.881340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dee7be-443a-421b-b0d2-b4d52924d08d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:54:59.915994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:54:59.916615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:54:59.916896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:54:59.917165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:54:59.928030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:54:59.933085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:54:59.934185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:54:59.936019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 540/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 31 03:54:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:54:59.940600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.038834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b7445-1f02-4d06-a819-53e5b451cc84 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.073469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.074058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.074331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.074641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.075326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:00.075622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.076982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:00.078352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:00.085893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 548/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:54:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:00.091837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.196143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd315a4-f2ed-47f4-8149-b2d6377b7ea8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.231860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.232871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.233147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.233418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.236051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:00.245140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.248121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:55:00.264403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:00.265421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:55:00.271432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:55:00.271432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:55:00.271432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:00.271432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 541/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:00.273137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.374999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f830d-cb56-4f05-9cb4-5b2a2d6e302e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.407666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.408254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.408521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.408785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.409414 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:00.409707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.411057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:00.422579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 549/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:00.428426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.536284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e3051-a478-4f40-8360-68d20831d250 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.571181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.571804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.572077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.572345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.580283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.581721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:55:00.592886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:00.593863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:00.595362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 542/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 31 03:55:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:00.599912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.659662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:00.705140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0dd4-4453-4ef0-9efe-9f7fa732b2ac None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.739979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.740570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.740844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.741111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.741829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:00.742134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.743589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d116075-596b-451d-9ee0-c665bab982a4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:00.761018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 550/1092] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 31 03:55:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:00.767098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:00.820157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:00.873981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f380da7-ff1e-4a7e-9634-1b83d1786ce7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:00.908978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:00.909565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:00.909902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:00.910109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:00.978632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:00.980120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.028301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.077360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31e1e-b9b1-4bf1-afd5-cbc9fe78364d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.077914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31e1e-b9b1-4bf1-afd5-cbc9fe78364d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.078006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31e1e-b9b1-4bf1-afd5-cbc9fe78364d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.079382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 551/1093] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:01.080949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 543/1094] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:01.084343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.084627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.084903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.086403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.102586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:01.103813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:01.171351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 552/1095] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:01.189297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724d95e-d3ee-4749-a43a-fc19c40e1eeb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.222865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.223512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.223789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.224057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.234009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.235494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.243806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:55:01.251194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:01.252311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:55:01.253418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:55:01.253418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:55:01.253418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:01.257097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 544/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:01.258306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.304672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.358447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6034e-d454-4723-8f5e-f319eed64204 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.392352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.392945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.393217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.393486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.394122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:01.394420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.395817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.406931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 553/1097] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:01.412660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.430958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.516943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba3ee-09cb-4838-8c9d-53d27943d980 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.531516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0baac-ebbc-4924-a726-f30c9cdf78e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.533813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.550557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.551209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.551491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.551767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.558291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.560019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.565087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.565702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.565975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.566222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:55:01.571469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:01.572519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:01.573822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 545/1098] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:01.578473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.579541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:01.584383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:01.585578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:55:01.586768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 554/1099] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:55:01.591487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.636840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.677929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaed6c0-1c63-471a-9de4-6032ed782ae2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.686608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008ac37-7c2a-4136-8c8e-916f29c282a8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.711672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.712289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.712561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.712824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.713503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:01.713803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.715290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.719823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.720421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.720697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.720968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.721630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:01.721914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.723327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.726797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 546/1100] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:01.736309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 555/1101] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:01.736465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.737690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.765007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.831662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834316-f730-492a-b0e6-addf3b24f246 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.834929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a39f-83ff-4304-b637-02c776e5c87b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:01.851387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.866153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.866885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.867441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.867937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.868067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:01.868677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:01.868955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:01.869225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] PATH_INFO: `/v3/users/647fff81c4cb46e082eb687705426dfd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:01.869314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] RBAC: Adding query filter params (role.id=28b5e44ef9574928be80ae944f02dd05, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=ed122ea299374f9cb2b8b4854df14d65) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:01.869762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.870654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] RBAC: Authorizing `identity:get_user(user_id=647fff81c4cb46e082eb687705426dfd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:01.872159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.872718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:01.873790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 556/1102] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/users/647fff81c4cb46e082eb687705426dfd => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:01.879547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:01.880563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 547/1103] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/role_assignments?user.id=ed122ea299374f9cb2b8b4854df14d65&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=28b5e44ef9574928be80ae944f02dd05 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:01.976445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:01.983710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b27d9c-faad-4794-bb3a-aca9ba8c222e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:02.021930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:02.022535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:02.022811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:02.023104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:02.090813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:02.092292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:02.181955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:02.190326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 557/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 03:55:02.196487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:02.302407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4c29-7baa-4c61-956f-5daac4859aa6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:02.335759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:02.336377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:02.336651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:02.336919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:02.338052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:02.339527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32ef0-56a1-40b9-b715-2abbf28816b5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:02.341245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 548/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:02 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:02.347589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:02.447141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e84727-f655-4870-bf23-25e2d9b1ea08 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:02.483531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:02.484138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:02.484410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:02.484677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:02.485403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6e7cb145a47348e189557322c98b0682, user.id=647fff81c4cb46e082eb687705426dfd) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:02.485708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:02.487157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd3b3-ff66-445c-bae2-938bce38b713 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:02.493167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 558/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:02 2022] GET /identity/v3/role_assignments?user.id=647fff81c4cb46e082eb687705426dfd&scope.project.id=6e7cb145a47348e189557322c98b0682&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:04.386629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51a8c-9beb-4dd8-b9ff-29f0abcb8f78 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:04.387617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51a8c-9beb-4dd8-b9ff-29f0abcb8f78 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:04.387941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51a8c-9beb-4dd8-b9ff-29f0abcb8f78 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:04.390125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 549/1107] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:04.398699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:04.399152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:04.399319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:04.426164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:04.427384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:04.483807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:04.500036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 559/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:04.766501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51a8c-9beb-4dd8-b9ff-29f0abcb8f78 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:04.827855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:04.882241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51a8c-9beb-4dd8-b9ff-29f0abcb8f78 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:04.918491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:04.919123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:04.919395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:04.919657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:04.920836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:04.922272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b9cd8-91c3-493a-a44e-f7b6a36c3d75 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:04.923869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 550/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:04.933067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:05.036765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c42eb-85e4-43ff-95c0-4ea5f0006b58 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:05.072390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:05.072998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:05.073274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:05.073541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:05.074198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:05.075914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8e93-ba84-469d-853a-4460e964f899 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:05.098950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 560/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:55:04 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:07.049051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6a5e-df7c-4505-8632-806674f7b8ca None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:07.049643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6a5e-df7c-4505-8632-806674f7b8ca None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:07.049643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6a5e-df7c-4505-8632-806674f7b8ca None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:07.050929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 551/1111] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:07.056413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:07.056850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:07.056940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:07.077305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:07.078486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:07.124890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:07.154936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 561/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:07.426692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f6a5e-df7c-4505-8632-806674f7b8ca None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:07.556920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f6a5e-df7c-4505-8632-806674f7b8ca None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:07.576926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:07.592885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:07.593506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:07.593783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:07.594055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] PATH_INFO: `/v3/roles/ee67389746c245038655ca9ddc941126` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:07.595516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] RBAC: Authorizing `identity:get_role(role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:07.596904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:07.598215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 552/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:07 2022] GET /identity/v3/roles/ee67389746c245038655ca9ddc941126 => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:07.606969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:07.709763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb71684-6f04-4158-a854-e9ff52a52019 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:07.744826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:07.745747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:07.746019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:07.746328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] PATH_INFO: `/v3/users/28ca02e9d32d4d6286c6e549e22e4760` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:07.759493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] RBAC: Authorizing `identity:get_user(user_id=28ca02e9d32d4d6286c6e549e22e4760)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:07.760935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:07.762670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 562/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:07 2022] GET /identity/v3/users/28ca02e9d32d4d6286c6e549e22e4760 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:07.769494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:07.879803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17b2f1-dc6e-4002-bb60-f54c950b321e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:07.914436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:07.915063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:07.915340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:07.915606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:07.972159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:07.986890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:07.988374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:08.092368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 553/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:08.098892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:08.131780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:08.201977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2dbe-8084-45c4-a70f-945239cfc691 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:08.239505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:08.240051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:08.240327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:08.240591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:08.241824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:08.243288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4de8b-5718-4518-bf76-8a774a6fdb6f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:08.244924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 563/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:08 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:08.252376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:08.344547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:08.357100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a283-01d3-4c80-be7a-9b948056d516 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:08.395770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:08.396357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:08.396628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:08.396892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:08.397629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=6e7cb145a47348e189557322c98b0682, user.id=28ca02e9d32d4d6286c6e549e22e4760) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:08.397931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:08.399390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94aec-275d-476c-8767-eaf2cc2b15f1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:08.405135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 554/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:08 2022] GET /identity/v3/role_assignments?user.id=28ca02e9d32d4d6286c6e549e22e4760&scope.project.id=6e7cb145a47348e189557322c98b0682&role.id=ee67389746c245038655ca9ddc941126 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:10.243616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66fee-f22d-4280-966f-f4c86c8dd270 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:10.244289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66fee-f22d-4280-966f-f4c86c8dd270 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:10.244289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66fee-f22d-4280-966f-f4c86c8dd270 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:10.245823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 564/1118] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:10.252337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:10.253102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:10.253266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:10.281216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:10.282431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:10.352492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 555/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:10.619484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66fee-f22d-4280-966f-f4c86c8dd270 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:10.731082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:10.735789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66fee-f22d-4280-966f-f4c86c8dd270 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:10.774012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:10.774621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:10.774895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:10.775185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] PATH_INFO: `/v3/roles/ee67389746c245038655ca9ddc941126` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:10.776673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] RBAC: Authorizing `identity:get_role(role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:10.778086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:10.779451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 565/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:10 2022] GET /identity/v3/roles/ee67389746c245038655ca9ddc941126 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:10.785587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:10.876561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:10.902990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb126c40-20c8-4c08-a77a-0fbd456c8a9a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:10.939934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:10.940552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:10.940833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:10.941105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] PATH_INFO: `/v3/users/28ca02e9d32d4d6286c6e549e22e4760` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:10.942441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] RBAC: Authorizing `identity:get_user(user_id=28ca02e9d32d4d6286c6e549e22e4760)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:10.943907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:10.945547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 556/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:10 2022] GET /identity/v3/users/28ca02e9d32d4d6286c6e549e22e4760 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:10.952675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:11.079243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d05f8-e24e-4cc4-be78-376a53dd303a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:11.114757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:11.115409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:11.115690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:11.115961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:11.117210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:11.118661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95252e16-608f-40a6-b9bf-e0da853d9ea5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:11.120643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 566/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:10 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:11.127146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:11.231985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d667e-ca29-458e-bb99-16419a618b1c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:11.267293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:11.267960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:11.268216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:11.268484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682/users/28ca02e9d32d4d6286c6e549e22e4760/roles/ee67389746c245038655ca9ddc941126` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:11.270828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e7cb145a47348e189557322c98b0682, user_id=28ca02e9d32d4d6286c6e549e22e4760, role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:11.272327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab92d73-985a-4c5f-a6cf-e7148359c712 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:11.290418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 557/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:55:11 2022] PUT /identity/v3/projects/6e7cb145a47348e189557322c98b0682/users/28ca02e9d32d4d6286c6e549e22e4760/roles/ee67389746c245038655ca9ddc941126 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:55:13.131643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0eea-1765-4057-b3fe-86f0e69a89ac None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:13.132221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0eea-1765-4057-b3fe-86f0e69a89ac None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:13.132221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0eea-1765-4057-b3fe-86f0e69a89ac None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:13.134217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 567/1124] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:13.142830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:13.143269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:13.143441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:13.166112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:13.167337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:13.170194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:13.242222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 558/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:13.505116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0eea-1765-4057-b3fe-86f0e69a89ac None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:13.544728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:13.613074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0eea-1765-4057-b3fe-86f0e69a89ac None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:13.650397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:13.651005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:13.651310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:13.651575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] PATH_INFO: `/v3/roles/ee67389746c245038655ca9ddc941126` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:13.653224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] RBAC: Authorizing `identity:get_role(role_id=ee67389746c245038655ca9ddc941126)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:13.654738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:13.656087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 568/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:13 2022] GET /identity/v3/roles/ee67389746c245038655ca9ddc941126 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:13.662165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:13.766884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc416c2-4e93-4959-8b33-7da4bc88ef98 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:13.805339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:13.806345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:13.806804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:13.807279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] PATH_INFO: `/v3/users/28ca02e9d32d4d6286c6e549e22e4760` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:13.809127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] RBAC: Authorizing `identity:get_user(user_id=28ca02e9d32d4d6286c6e549e22e4760)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:13.810682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:13.812315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 559/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:13 2022] GET /identity/v3/users/28ca02e9d32d4d6286c6e549e22e4760 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:13.818946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:13.931690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1dd30-2857-43a7-b006-8d644cbffa69 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:13.934408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:13.967804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:13.968400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:13.968674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:13.968937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:14.038979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:14.040381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:14.139366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 569/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:14.145923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:14.157274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:14.247153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef6bad-5ac6-4936-8749-35b3ec60d015 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:14.283353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:14.283979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:14.284255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:14.284529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] PATH_INFO: `/v3/projects/6e7cb145a47348e189557322c98b0682` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:14.286317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=6e7cb145a47348e189557322c98b0682)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:14.287816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6ec7c-89e3-493e-a86f-b75decac5ca7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:14.289561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 560/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:14 2022] GET /identity/v3/projects/6e7cb145a47348e189557322c98b0682 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:14.296259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:14.299448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:14.407853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899ee37-9802-41e1-92c2-0ea0762ac4e8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:14.443943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:14.444537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:14.444807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:14.445073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:14.445788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] RBAC: Adding query filter params (role.id=ee67389746c245038655ca9ddc941126, scope.project.id=6e7cb145a47348e189557322c98b0682, user.id=28ca02e9d32d4d6286c6e549e22e4760) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:14.446098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:14.447558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9df59-ebc0-4d56-8d3b-1a2e6af9c094 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:14.454225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 570/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:14 2022] GET /identity/v3/role_assignments?user.id=28ca02e9d32d4d6286c6e549e22e4760&scope.project.id=6e7cb145a47348e189557322c98b0682&role.id=ee67389746c245038655ca9ddc941126 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:16.377558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396063cf-f49f-4d1e-83bd-a67bbe9f97cb None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:16.378452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396063cf-f49f-4d1e-83bd-a67bbe9f97cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:16.378452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396063cf-f49f-4d1e-83bd-a67bbe9f97cb None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:16.380664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 561/1131] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:16.386390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:16.386825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:16.387164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:16.406049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:16.407270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:16.472050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:16.477869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 571/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:16.747742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396063cf-f49f-4d1e-83bd-a67bbe9f97cb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:16.851164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:16.878872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396063cf-f49f-4d1e-83bd-a67bbe9f97cb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:16.916965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:16.917608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:16.917911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:16.918350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:16.919539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:16.920964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f537e89-8edc-46fc-aeba-d1a727fcd79d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:16.922525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 562/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:55:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:16.928697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:17.029930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf26f-30a8-4567-afb9-6ae4c01b5aed None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:17.064800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:17.065388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:17.065657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:17.065924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:17.066558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:17.068038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bb548-da11-473c-914c-e04f9daf2a64 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:17.102729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 572/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:55:16 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:18.992333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871c483-e2fd-4053-9928-9c059263856f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:18.992895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871c483-e2fd-4053-9928-9c059263856f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:18.992895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871c483-e2fd-4053-9928-9c059263856f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:18.994200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 563/1135] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:19.000919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:19.001668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:19.001817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:19.030463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:19.031665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:19.080763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:19.102455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 573/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:19.374633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871c483-e2fd-4053-9928-9c059263856f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:19.485051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871c483-e2fd-4053-9928-9c059263856f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:19.521594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:19.522202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:19.522477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:19.522741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:19.524008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:19.525449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73926-8d29-4407-b1d1-f57842635e0a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:19.527615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 564/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:55:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:19.533622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:19.640774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c9ede-374d-4a55-ba88-cafb9aee0e12 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:19.675123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:19.675725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:19.676008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:19.676279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:19.676925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:19.678366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c3d51-bba1-4391-b980-e1ee60d7af73 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:19.707391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 574/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:55:19 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:21.593196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ba7fc-a10b-467c-a8bf-80419bbb99a3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:21.594037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ba7fc-a10b-467c-a8bf-80419bbb99a3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:21.594037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ba7fc-a10b-467c-a8bf-80419bbb99a3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:21.596358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 565/1139] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:21.602545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:21.603309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:21.603489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:21.623538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:21.624745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:21.659576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:21.709116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 575/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:21.984705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ba7fc-a10b-467c-a8bf-80419bbb99a3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:22.005373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:22.100372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ba7fc-a10b-467c-a8bf-80419bbb99a3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:22.135776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:22.136382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:22.136651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:22.136913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:22.146870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:22.151633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:22.152860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:22.153778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 566/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:22.159811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:22.269920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e83-24ee-411d-ad29-13201d92f26e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:22.316297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:22.316905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:22.317186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:22.317456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:22.318123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:22.318420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:22.319841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:22.328409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 576/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:22.334495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:22.425105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:22.463068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd46b47-b8e8-4c24-aa64-7a6d9195bf14 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:22.502246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:22.502869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:22.503176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:22.503441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] PATH_INFO: `/v3/users/153975a237ff47dd82b7428c90faf5f5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:22.518259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] RBAC: Authorizing `identity:get_user(user_id=153975a237ff47dd82b7428c90faf5f5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:22.519740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:22.521450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 567/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:22 2022] GET /identity/v3/users/153975a237ff47dd82b7428c90faf5f5 => generated 286 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:22.531431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:22.566805 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:22.654588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5567fd2-8569-4ed3-9dcc-f26d1e83edae None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:22.689862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:22.690450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:22.690723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:22.690990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:22.766338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:22.767811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:22.801207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:22.879524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 577/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:22.885815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:22.989301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238acb5-3db7-4f1b-b3a0-a684ab27cf27 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:23.020673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:23.027665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:23.028280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:23.028553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:23.028815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] PATH_INFO: `/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:23.029930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] RBAC: Authorizing `identity:get_project(project_id=1c8f3457fdb9476ebefd8466b6af2a8f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:23.031387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855919-12b6-4b85-bbc5-b3acba07a183 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:23.033011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 568/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:22 2022] GET /identity/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:23.039663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:23.142073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982476c-2b83-453c-8ca4-0d2d1c97366c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:23.169368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:23.179164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:23.179775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:23.180050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:23.180303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:23.181014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=1c8f3457fdb9476ebefd8466b6af2a8f, user.id=153975a237ff47dd82b7428c90faf5f5) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:23.181314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:23.182743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8b492-c12c-4275-b685-fc62d0a1117c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:23.189049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 578/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:23 2022] GET /identity/v3/role_assignments?user.id=153975a237ff47dd82b7428c90faf5f5&scope.project.id=1c8f3457fdb9476ebefd8466b6af2a8f&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:25.022874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1778-a6de-409a-bc4f-1b1069718541 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:25.023478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1778-a6de-409a-bc4f-1b1069718541 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:25.023478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d1778-a6de-409a-bc4f-1b1069718541 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:25.024781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 569/1147] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:25.031447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:25.032166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:25.032330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:25.061576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:25.062751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:25.132912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 579/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:25.402355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d1778-a6de-409a-bc4f-1b1069718541 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:25.511355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d1778-a6de-409a-bc4f-1b1069718541 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:25.546870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:25.547442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:25.547716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:25.547977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:25.558637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:25.563426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:25.564456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:25.565738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 570/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:25.573947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:25.676458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2919333-6f39-4a3c-a6a6-8481f3b29993 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:25.700220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:25.713060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:25.713664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:25.713934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:25.714355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:25.715018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:25.715350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:25.716820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:25.725280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 580/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:25.731418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:25.791883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:25.840694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b6590-fef0-477e-846d-17e56b63cb55 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:25.877140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:25.877749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:25.878021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:25.878281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] PATH_INFO: `/v3/users/153975a237ff47dd82b7428c90faf5f5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:25.879541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=153975a237ff47dd82b7428c90faf5f5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:25.880966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:25.882521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 571/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:25 2022] GET /identity/v3/users/153975a237ff47dd82b7428c90faf5f5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:25.889174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:26.010953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef23de-e1b0-4227-a774-51c08edb7f05 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:26.048377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:26.048980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:26.049259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:26.049520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] PATH_INFO: `/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:26.050768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] RBAC: Authorizing `identity:get_project(project_id=1c8f3457fdb9476ebefd8466b6af2a8f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:26.052224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f51740-0cd5-4d1c-be63-bc838430dd02 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:26.053857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 581/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:25 2022] GET /identity/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:26.061776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:26.169144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e2c7-03e1-4096-9e7d-5f65883b8cf6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:26.194935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:26.205288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:26.205926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:26.206194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:26.206455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] PATH_INFO: `/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f/users/153975a237ff47dd82b7428c90faf5f5/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:26.208414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c8f3457fdb9476ebefd8466b6af2a8f, user_id=153975a237ff47dd82b7428c90faf5f5, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:26.209861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acced9-c46b-46ac-ba22-6b281f509872 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:26.224774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 572/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:55:26 2022] PUT /identity/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f/users/153975a237ff47dd82b7428c90faf5f5/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:55:28.072870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140b5b-0b4b-4a81-8c0a-0de3c2fecca2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:28.073938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140b5b-0b4b-4a81-8c0a-0de3c2fecca2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:28.073938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140b5b-0b4b-4a81-8c0a-0de3c2fecca2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:28.075980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 582/1154] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:28.084768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:28.085450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:28.085635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:28.106417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:28.107648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:28.179254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 573/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:28.445150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75140b5b-0b4b-4a81-8c0a-0de3c2fecca2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:28.502525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:28.554968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75140b5b-0b4b-4a81-8c0a-0de3c2fecca2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:28.591638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:28.592247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:28.592520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:28.592784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:28.603307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:28.608162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:28.609215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:55:28.610259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 583/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:28.615591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:28.738080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0c370-d304-4a1a-9c65-4fd3828e9fa6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:28.755895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:28.775985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:28.776662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:28.777065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:28.777355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:28.778020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:28.778321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:28.779746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:28.787787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 574/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:55:28.793935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:28.906474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd541-3096-4514-bef8-a2c96f49b376 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:28.919835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:28.942905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:28.943549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:28.943830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:28.944106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] PATH_INFO: `/v3/users/153975a237ff47dd82b7428c90faf5f5` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:28.945340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] RBAC: Authorizing `identity:get_user(user_id=153975a237ff47dd82b7428c90faf5f5)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:28.946775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:28.948407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 584/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:28 2022] GET /identity/v3/users/153975a237ff47dd82b7428c90faf5f5 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:28.955321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:29.068221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c30d71-ee81-465d-b205-14fd6ecd18e0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:29.105406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:29.105993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:29.106109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:29.106409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:29.152827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:29.176007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:29.177461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:29.284297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 575/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:29.293207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:29.399443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a870ef4-66ce-4824-8a45-3f72d2b5747f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:29.447213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:29.447835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:29.448116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:29.448454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] PATH_INFO: `/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:29.449628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] RBAC: Authorizing `identity:get_project(project_id=1c8f3457fdb9476ebefd8466b6af2a8f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:29.451066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed24ec7-12e4-4cf0-a478-9ef2460723f1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:29.452738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 585/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:29 2022] GET /identity/v3/projects/1c8f3457fdb9476ebefd8466b6af2a8f => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:29.459066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:29.544141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:29.571936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275a2-4b2e-43f9-a024-a92276471b24 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:29.608766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:29.609352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:29.609626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:29.609899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:29.610625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=1c8f3457fdb9476ebefd8466b6af2a8f, user.id=153975a237ff47dd82b7428c90faf5f5) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:29.610936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:29.612401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dac36-6608-4bde-aa5d-f2e563c13330 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:29.618228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 576/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:29 2022] GET /identity/v3/role_assignments?user.id=153975a237ff47dd82b7428c90faf5f5&scope.project.id=1c8f3457fdb9476ebefd8466b6af2a8f&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:31.521081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e442-1e34-4860-8751-75cdae0d593e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:31.521960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e442-1e34-4860-8751-75cdae0d593e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:31.521960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e442-1e34-4860-8751-75cdae0d593e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:31.524184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 586/1162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:31.530528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160795-528a-4988-bae9-49ec18122475 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:31.530974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160795-528a-4988-bae9-49ec18122475 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:31.531114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160795-528a-4988-bae9-49ec18122475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:31.554459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-78160795-528a-4988-bae9-49ec18122475 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:31.555624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-78160795-528a-4988-bae9-49ec18122475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:31.625412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 577/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:31.890441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e442-1e34-4860-8751-75cdae0d593e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:31.951765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:32.012494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e442-1e34-4860-8751-75cdae0d593e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:32.049495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:32.050339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:32.050714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:32.051103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:32.138816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:32.140305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:32.189277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:32.253549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 587/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:32.260117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160795-528a-4988-bae9-49ec18122475 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:32.334475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:32.379770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160795-528a-4988-bae9-49ec18122475 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:32.418249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:32.418877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:32.419203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:32.419479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:32.426398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:32.427885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-64ff38eb-45a3-4277-a152-0a29fbf3e5d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:32.439549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:32.440595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:32.441507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 578/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:55:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:32.447182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:32.552979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58296cc5-4b75-43f3-832d-3ecbf1803a84 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:32.589691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:32.590287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:32.590583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:32.590846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:32.591516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:32.591813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:32.593184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5a927-0d40-4cc9-a4d2-2e93ced7da8e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:32.600259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 588/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 31 03:55:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:34.465956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453697f-45a0-423c-9004-dad2076e7730 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:34.466845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453697f-45a0-423c-9004-dad2076e7730 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:34.466845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453697f-45a0-423c-9004-dad2076e7730 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:34.468991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 579/1167] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:34.475367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d4b8-e3b3-4ad9-ab01-3252ed60c568 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:34.475808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d4b8-e3b3-4ad9-ab01-3252ed60c568 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:34.475953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3d4b8-e3b3-4ad9-ab01-3252ed60c568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:34.496648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-65e3d4b8-e3b3-4ad9-ab01-3252ed60c568 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:34.497849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-65e3d4b8-e3b3-4ad9-ab01-3252ed60c568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:34.505412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:34.569920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 589/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:34.833157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453697f-45a0-423c-9004-dad2076e7730 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:34.949123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453697f-45a0-423c-9004-dad2076e7730 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:34.979130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:34.986365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:34.986956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:34.987259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:34.987550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:34.988191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:34.989568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cfbf-7d89-4b90-a9ef-5feec48a52c3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:35.007454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 580/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Aug 31 03:55:34 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:36.924081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac83d3-0f9d-4790-93d1-e8316cdb295a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:36.924934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac83d3-0f9d-4790-93d1-e8316cdb295a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:36.924934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac83d3-0f9d-4790-93d1-e8316cdb295a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:36.927077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 590/1170] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:36.935528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:36.935995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:36.936098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:36.955803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:36.956994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:37.027233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 581/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:37.302952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac83d3-0f9d-4790-93d1-e8316cdb295a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:37.410689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac83d3-0f9d-4790-93d1-e8316cdb295a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:37.445436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:37.446052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:37.446320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:37.446592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] PATH_INFO: `/v3/domains/48744189d73840938b5537e46081a3db` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:37.457569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=48744189d73840938b5537e46081a3db)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:37.459004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fc625-1ca3-4478-bf75-041287eac2fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:37.460619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 591/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 31 03:55:37 2022] GET /identity/v3/domains/48744189d73840938b5537e46081a3db => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:37.467067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:37.575327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36c5db-21c8-4aed-82ef-d30fffc49c53 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:37.599197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:37.612244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:37.612838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:37.613122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:37.613395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:37.614073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:37.615576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63984b-f772-4d21-bffc-9b4dd302b489 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:37.646394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 582/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:55:37 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:39.583074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b323a-f8e4-4dbd-95ac-781236bd861e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:39.583889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b323a-f8e4-4dbd-95ac-781236bd861e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:39.583889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b323a-f8e4-4dbd-95ac-781236bd861e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:39.585991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 592/1174] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:39.592618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:39.593090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:39.593195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:39.616114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:39.617293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:39.690538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 583/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:39.957046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b323a-f8e4-4dbd-95ac-781236bd861e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:39.960491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:40.083597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b323a-f8e4-4dbd-95ac-781236bd861e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:40.119259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:40.120490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:40.120804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:40.121082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] PATH_INFO: `/v3/domains/48744189d73840938b5537e46081a3db` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:40.122244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=48744189d73840938b5537e46081a3db)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:40.123732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03773d-791c-48bf-9a66-91e8ba72ae92 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:40.125305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 593/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 31 03:55:39 2022] GET /identity/v3/domains/48744189d73840938b5537e46081a3db => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:40.134139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:40.246068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c75fd-9799-4d7d-8307-077ffd13c48a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:40.248498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:40.291713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:40.292164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:40.292457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:40.292713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:40.293353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:40.294870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49bee2-3171-44d4-ba4b-a1f8593f6f18 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:40.317075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 584/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:55:40 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:55:42.208336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cfb75-5444-4b03-abf6-9a5c0c34b4aa None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:42.209271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cfb75-5444-4b03-abf6-9a5c0c34b4aa None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:42.209271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cfb75-5444-4b03-abf6-9a5c0c34b4aa None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:42.211448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 594/1178] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:42.217646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:42.218095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:42.218192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:42.241223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:42.242406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:42.315809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 585/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:42.582235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cfb75-5444-4b03-abf6-9a5c0c34b4aa None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:42.658132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:42.691359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cfb75-5444-4b03-abf6-9a5c0c34b4aa None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:42.735502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:42.736123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:42.736395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:42.736663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:42.747264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:42.752034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:42.753094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:55:42.754076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 595/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:42.759334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:42.855558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:42.872738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b22a3-16d1-4d58-a2aa-c8b10b624b32 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:42.912872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:42.913469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:42.913739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:42.914006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:42.914655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:42.914963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:42.916379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:42.924660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 586/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:42.931279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:43.058141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a3d48-fbd6-4636-97ca-296c4a67b7e1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:43.087618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:43.094971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:43.095628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:43.095906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:43.096171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] PATH_INFO: `/v3/users/4eebf4978b0344a287a3ef10c62f5300` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:43.111126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] RBAC: Authorizing `identity:get_user(user_id=4eebf4978b0344a287a3ef10c62f5300)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:43.118532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:43.120990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 596/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:42 2022] GET /identity/v3/users/4eebf4978b0344a287a3ef10c62f5300 => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:43.130325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:43.240408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633cc6d-fccf-4b86-9d42-301caf7bcffd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:43.276593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:43.277197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:43.277471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:43.277741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:43.300190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:43.347584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:43.349030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:43.472792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 587/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:43.482042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:43.597028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2fdfe-4aef-45bb-ac5a-08909f46337c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:43.636117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:43.636726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:43.637000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:43.637268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] PATH_INFO: `/v3/projects/db56547eb99b444ab605953cc1731fec` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:43.638491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] RBAC: Authorizing `identity:get_project(project_id=db56547eb99b444ab605953cc1731fec)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:43.639953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66cfa6-5f53-48e8-80cd-392eb6eda7bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:43.641577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 597/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:43 2022] GET /identity/v3/projects/db56547eb99b444ab605953cc1731fec => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:43.648499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:43.708879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:43.762545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b594438-acbb-4192-b40a-792fa17f67e8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:43.805139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:43.805732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:43.806003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:43.806273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:43.806990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=db56547eb99b444ab605953cc1731fec, user.id=4eebf4978b0344a287a3ef10c62f5300) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:43.807786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:43.809230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373755a-ecd5-48e8-b454-9fb5e4918515 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:43.814763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 588/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:43 2022] GET /identity/v3/role_assignments?user.id=4eebf4978b0344a287a3ef10c62f5300&scope.project.id=db56547eb99b444ab605953cc1731fec&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:45.672814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08b6a-8be1-46bd-9c00-2a1ecd6a4829 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:45.673671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08b6a-8be1-46bd-9c00-2a1ecd6a4829 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:45.673671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08b6a-8be1-46bd-9c00-2a1ecd6a4829 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:45.675718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 598/1186] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:45.681070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:45.681507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:45.681613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:45.701367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:45.702544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:45.770833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 589/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:46.033713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08b6a-8be1-46bd-9c00-2a1ecd6a4829 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:46.100148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:46.160392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08b6a-8be1-46bd-9c00-2a1ecd6a4829 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:46.196408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:46.197072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:46.197349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:46.197714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:46.208870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:46.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:46.214971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:55:46.215844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 599/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:46.223932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:46.279174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:46.343066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb654a2-45c7-431e-9fdf-d39cd44923de None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:46.389639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:46.390232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:46.390502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:46.390763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:46.391455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:46.391756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:46.393125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:46.401606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 590/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:46.407899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:46.516563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a360271-e5b2-4b81-ab7e-a13e31dc3064 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:46.534208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:46.553631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:46.554180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:46.554450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:46.554716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] PATH_INFO: `/v3/users/4eebf4978b0344a287a3ef10c62f5300` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:46.555916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] RBAC: Authorizing `identity:get_user(user_id=4eebf4978b0344a287a3ef10c62f5300)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:46.557387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:46.558980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 600/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:46 2022] GET /identity/v3/users/4eebf4978b0344a287a3ef10c62f5300 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:46.568673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:46.675590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5494e3a-6361-4736-ae90-f815a013ade7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:46.684685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:46.716371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:46.716982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:46.717261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:46.717530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] PATH_INFO: `/v3/projects/db56547eb99b444ab605953cc1731fec` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:46.718649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] RBAC: Authorizing `identity:get_project(project_id=db56547eb99b444ab605953cc1731fec)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:46.720136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f51b52-9682-474b-ac6f-f818e7da9e35 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:46.721732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 591/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:46 2022] GET /identity/v3/projects/db56547eb99b444ab605953cc1731fec => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:46.727968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:46.840307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c492c-5ea8-4f24-b47c-4feff2de513d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:46.879652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:46.880291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:46.880549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:46.880829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] PATH_INFO: `/v3/projects/db56547eb99b444ab605953cc1731fec/users/4eebf4978b0344a287a3ef10c62f5300/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:46.882854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] RBAC: Authorizing `identity:create_grant(project_id=db56547eb99b444ab605953cc1731fec, user_id=4eebf4978b0344a287a3ef10c62f5300, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:46.884321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad1af0-58ba-49d7-848a-118e5541c561 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:46.904244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 601/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:55:46 2022] PUT /identity/v3/projects/db56547eb99b444ab605953cc1731fec/users/4eebf4978b0344a287a3ef10c62f5300/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:55:48.742096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8091-a9cc-44d0-a594-743008030b34 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:48.742662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8091-a9cc-44d0-a594-743008030b34 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:48.742662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8091-a9cc-44d0-a594-743008030b34 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:48.745170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 592/1193] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:55:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:55:48.753794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:48.754206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:48.754377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:48.775851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:55:48.777055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:55:48.818770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:48.847532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 602/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:55:49.107928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff8091-a9cc-44d0-a594-743008030b34 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:49.229130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff8091-a9cc-44d0-a594-743008030b34 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:49.267779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:49.268923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:49.269358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:49.269783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:49.285400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:55:49.290160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:55:49.291504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:55:49.293057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 593/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:55:49.297491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:49.400968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f1bcc-e0bf-45cd-b93f-c6eb7717d24d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:49.437596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:49.438186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:49.438464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:49.438727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:49.439413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:49.439705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:49.441064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:49.449144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 603/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:49.455327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:49.491627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:49.558718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e83ddc-0fd9-420f-a68a-40ab1446d598 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:49.594573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:49.595222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:49.595500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:49.595768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] PATH_INFO: `/v3/users/4eebf4978b0344a287a3ef10c62f5300` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:49.597020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] RBAC: Authorizing `identity:get_user(user_id=4eebf4978b0344a287a3ef10c62f5300)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:49.598458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:49.600083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 594/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 31 03:55:49 2022] GET /identity/v3/users/4eebf4978b0344a287a3ef10c62f5300 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:49.607585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:49.645464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:49.726176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300af53c-f0fa-4d05-ac72-211e9bf4887e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:49.762819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:49.763620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:49.764052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:49.764355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:49.845015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:49.846801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:49.888427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:49.968790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 604/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:55:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:55:49.976324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:50.101349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eace9-c53a-426c-a196-7aa68c779c15 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:50.104968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:50.144051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:50.144703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:50.144979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:50.145243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] PATH_INFO: `/v3/projects/db56547eb99b444ab605953cc1731fec` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:50.146376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] RBAC: Authorizing `identity:get_project(project_id=db56547eb99b444ab605953cc1731fec)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:50.147838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642666d-94ab-4541-94c8-53ae58cb0591 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:50.149521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 595/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 31 03:55:49 2022] GET /identity/v3/projects/db56547eb99b444ab605953cc1731fec => generated 361 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:55:50.160984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:55:50.274379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73a441-dfa1-4054-8073-1d652835b36b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:55:50.317587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:55:50.318192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:55:50.318467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:55:50.318741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:55:50.319491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=db56547eb99b444ab605953cc1731fec, user.id=4eebf4978b0344a287a3ef10c62f5300) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:55:50.320071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:55:50.321505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dda3c-0410-4c79-bfe4-3e6f5c016c56 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:55:50.324917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:55:50.330388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 605/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:55:50 2022] GET /identity/v3/role_assignments?user.id=4eebf4978b0344a287a3ef10c62f5300&scope.project.id=db56547eb99b444ab605953cc1731fec&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:00.138285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=14 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:45.429273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdf40-273a-4465-b000-59e0cf176f24 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:45.430275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdf40-273a-4465-b000-59e0cf176f24 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:45.430275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdf40-273a-4465-b000-59e0cf176f24 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:45.431328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 596/1201] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:56:45.438251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:45.438780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:45.438870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:45.458778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:56:45.460101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:56:45.553215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 606/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:56:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:56:45.839768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fdf40-273a-4465-b000-59e0cf176f24 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:45.977609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fdf40-273a-4465-b000-59e0cf176f24 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:46.020129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:46.020910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:46.021200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:46.021544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:46.036761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:46.040760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:46.055726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:46.056867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:46.063131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:46.063131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:46.063131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:46.063131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 597/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:56:46.065131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:46.190864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837742e-5fbd-47eb-9c46-6b652ddecbbd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:46.225643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:46.232759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:46.233372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:46.233687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:46.233905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:46.234580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:46.234937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:46.236445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71db75-7c8b-4b69-a68b-00aae1c7fc6d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:46.257379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 607/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:46.263756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:46.339551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:46.398551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170722b-f0be-4300-ae53-73073c61967a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:46.439094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:46.439094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:46.439965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:46.439965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:46.440481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:46.442335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48886a-a54d-4f5d-a2b3-07a586d92aee None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:46.484554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 598/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 31 03:56:46 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:56:48.586126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009e31-e4bf-4e3d-a6f8-c5d987fb53cd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:48.586650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009e31-e4bf-4e3d-a6f8-c5d987fb53cd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:48.586650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009e31-e4bf-4e3d-a6f8-c5d987fb53cd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:48.587967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 608/1206] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:56:48.593258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:48.593802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:48.593899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:48.612846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:56:48.614047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:56:48.691399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 599/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:56:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:56:48.961215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009e31-e4bf-4e3d-a6f8-c5d987fb53cd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:49.082432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009e31-e4bf-4e3d-a6f8-c5d987fb53cd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:49.124104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:49.124749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:49.125031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:49.125302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:49.136701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:56:49.141551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:49.142699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:49.143794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 609/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:56:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:49.149054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:49.192549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:49.259512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e73dc-cc61-4ace-a807-31c26b277512 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:49.300939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:49.301528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:49.301806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:49.302070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:49.302725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:49.303065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:49.304568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:49.313751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 600/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:56:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:56:49.319572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:49.379930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:49.436766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dbecc-892f-4749-857a-2f1fd616c44d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:49.480151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:49.481628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:49.481628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:49.481628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:49.494141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:49.496029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:49.512991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:49.514281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:49.515240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:49.515240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:49.515240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:49.515240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 610/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:49.520222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:49.635909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:49.642432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7239cf-e0e1-4905-a0ea-2833d7b4a85d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:49.691236 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:49.692153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:49.692153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:49.692415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:49.693112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:49.693431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:49.694941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:49.709103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 601/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:49.715460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:49.838609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260bfda-917d-420d-8afb-72f60f6fc76c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:49.842197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:49.886379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:49.887122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:49.887406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:49.887736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:49.905291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:49.907856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:56:49.926867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:49.929328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:49.930410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 611/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:49.936189 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:50.082914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538f78-11c4-4e53-9455-30bc10a7396c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:50.126656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:50.127650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:50.127999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:50.128277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:50.129214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:50.129642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:50.131171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:50.153757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 602/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:56:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:50.160093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:50.292836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5e6d3-40a7-4c51-a0c2-58683877a528 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:50.336256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:50.337055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:50.337391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:50.337675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:50.376632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:50.424241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:50.426017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:50.553259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 612/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:56:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:56:50.559124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:50.584314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:50.685549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc530d-48d4-4a82-9645-59d289d3ad84 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:50.729180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:50.730901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:50.731347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:50.731622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:50.747460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:50.750350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:50.770133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:50.771290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:50.772168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:50.772168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:50.772168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:50.772168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 603/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:56:50.778885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:50.854475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:50.900276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0434221-4156-4f23-a19f-9a7a755f229e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:50.942979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:50.943534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:50.943809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:50.944073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:50.944791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:50.945389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:50.946806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:50.961909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 613/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:50.967500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:51.052361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:51.090301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69aacb-f48a-4c47-b8c8-194e14298db2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:51.128971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:51.129621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:51.129901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:51.130224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:51.138424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:51.140015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:56:51.155625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:51.157051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:51.164547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 604/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:51.165086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:51.290832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424afba8-124f-4ad6-9e4a-aca6c7a40e28 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:51.315849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:51.328981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:51.329635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:51.329914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:51.330185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:51.330880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:51.331231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:51.332746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801a03-0d05-4857-b6b1-2b5996909518 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:51.351107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 614/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:56:51.353228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:51.463251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833516c-1cbf-4313-b8d8-fa67bc954094 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:51.497440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:51.505182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:51.505903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:51.506172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:51.506507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:51.507476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=bd2e18144a794d36b3d81cd3c8ef9cc3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:51.507740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:51.509179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5b0e4-ed2a-4561-abbd-29146ea20fa3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:51.515484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 605/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:56:51 2022] GET /identity/v3/role_assignments?user.id=bd2e18144a794d36b3d81cd3c8ef9cc3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:53.481293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b87e-e302-41b7-ae61-9e296862956a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:53.481875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b87e-e302-41b7-ae61-9e296862956a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:53.481875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b87e-e302-41b7-ae61-9e296862956a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:53.484489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 615/1220] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:56:53.488542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:53.488990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:53.489082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:53.508861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:56:53.510056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:56:53.583721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 606/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:56:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:56:53.873748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602b87e-e302-41b7-ae61-9e296862956a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:53.980259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602b87e-e302-41b7-ae61-9e296862956a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.015234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.015859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.016137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.016448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.026751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:56:54.031626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:54.032801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:54.033765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 616/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:56:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:54.039065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:54.144704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d561f1-b898-4822-8ac9-81c22c90cef0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.168212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:54.180272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.180948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.181229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.181566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.182258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:54.182610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:54.184234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:54.193499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 607/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:54.199572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:54.245818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:54.309964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba409dd-fbd5-44cc-a955-464c9ebcbbfa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.343960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.344586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.344866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.345133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.355389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:54.356843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:54.373204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:54.379077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:54.380206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:54.380206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:54.380206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:54.380206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 617/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:56:54.380481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:54.483776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95260bc-9eff-4237-9e2f-40a1413c316e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.521607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.522335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.522638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.523203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.523926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:54.524274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:54.525874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:54.540089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 608/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:54.546405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:54.648289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:54.655932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114452-3646-4243-bec8-47fe9bf5326b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.705457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.706069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.706366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.706616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.716365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:54.717828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:56:54.731887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:54.733479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:54.735014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 618/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:54.739443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:54.798911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:54.866041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aadc68-2d8c-45f1-b500-aad130750786 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:54.906985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:54.907823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:54.908290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:54.908560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:54.909494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:54.909872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:54.911876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:54.933570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 609/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:54.939763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:55.052016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec19471-d60b-461f-a05e-127fe44cabf6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:55.089647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:55.090311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:55.090668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:55.091106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:55.103460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:55.104905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:55.119895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:55.121159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:55.122226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:55.122226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:55.122226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:55.122226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 619/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:56:55.127252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:55.270655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdab86-d814-43cb-ac3a-44e8fee320c3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:55.273114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:55.313423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:55.313945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:55.314225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:55.314484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:55.315153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:55.315452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:55.316958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:55.329231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 610/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:56:55.335747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:55.345705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:55.448054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fe4a7-b971-44c9-8e2b-504e5c6f30ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:55.489798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:55.490415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:55.490688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:55.490950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:55.498083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:55.499655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:56:55.513649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:55.514839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:55.515831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 620/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:56:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:55.521209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:55.638860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3a969-5598-435a-8c98-a53e29687ad6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:55.676634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:55.677244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:55.677539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:55.677783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:55.678503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:55.678820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:55.680311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea5b6b-7a6d-4ee4-a902-8e8197c027a6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:55.692806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 611/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:56:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:55.699152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:55.776769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:55.813711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89e8a-2af1-4cbc-b957-a9a410d1e472 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:55.849787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:55.850607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:55.850893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:55.851237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/bd2e18144a794d36b3d81cd3c8ef9cc3/roles/3df158de7ee04605945a4d8e51c9b24e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:55.866693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=bd2e18144a794d36b3d81cd3c8ef9cc3, role_id=3df158de7ee04605945a4d8e51c9b24e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:55.868503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1186c4b-92f7-4543-aa8e-ea2afc916c86 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:55.883872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 621/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:56:55 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/bd2e18144a794d36b3d81cd3c8ef9cc3/roles/3df158de7ee04605945a4d8e51c9b24e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:56:57.869933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb876b-c76e-4abb-aab7-74468266f5a5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:57.870522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb876b-c76e-4abb-aab7-74468266f5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:57.870522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb876b-c76e-4abb-aab7-74468266f5a5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:57.871849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 612/1233] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:56:57.876948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:57.877374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:57.877516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:57.896132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:56:57.897473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:56:57.966112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 622/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:56:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:56:58.229942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb876b-c76e-4abb-aab7-74468266f5a5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:58.258093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:58.343511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb876b-c76e-4abb-aab7-74468266f5a5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:58.381339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:58.382142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:58.382423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:58.382778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:58.393783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:56:58.398861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:58.399995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:58.401006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 613/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 31 03:56:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:58.406658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:58.452940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:58.505697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda8995-f57c-45a0-81e4-9705e41aad8b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:58.543004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:58.543641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:58.543934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:58.544192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:58.544858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:58.545189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:58.546562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:58.555130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 623/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 31 03:56:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:58.560930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:58.649834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:58.669009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae2cce-7080-470a-81c9-20f11f60d231 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:58.707172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:58.707806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:58.708087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:58.708438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:58.721231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:58.722725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:58.740064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:58.746824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:58.747954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:58.747954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:58.747954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:58.747954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 614/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:58.748246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:58.859396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262044-1a1f-410d-9322-78f55e5fd67c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:58.870332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:58.898838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:58.899568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:58.899884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:58.900206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:58.900953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:58.901313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:58.902826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:58.915755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 624/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:56:58.921533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:59.028670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cbb0-efd1-4361-93cf-52cf8be9538b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:59.063235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:59.063928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:59.064257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:59.064491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:59.067253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:59.072837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:59.074452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:56:59.086434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:59.087590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:56:59.088842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:56:59.092426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 615/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:59.093428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:59.201443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f448-97f0-43d9-b756-460d581b3eb2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:59.247882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:59.248630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:59.248918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:59.249261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:59.250000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:59.250367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:59.251946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:59.273924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 625/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:56:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:56:59.280352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:59.385185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ff5b-92e4-4fc4-bd13-54c987888c30 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:59.421137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:59.421760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:59.422039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:59.422317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:59.484882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:59.497062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:59.499369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:59.630748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 616/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:56:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:56:59.633948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:59.666634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:59.771600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae3bff-1311-45c7-9e2f-9e40f1b1a6fa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:59.807891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:59.808517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:59.808796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:59.809068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:59.819303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:59.821467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:56:59.837010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:56:59.838146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:56:59.839093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:56:59.839093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:56:59.839093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:56:59.839093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 626/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:56:59.844061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:56:59.952966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:56:59.959065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37184d9b-057a-4239-88a8-24a5c421a13e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:56:59.996253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:56:59.996850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:56:59.997151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:56:59.997385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:56:59.998021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:56:59.998339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:56:59.999858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:00.012147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 617/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:00.018151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:00.101251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:00.124749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dddc64-b959-4ef9-9d2e-e406f1fb1b9d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:00.160632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:00.161256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:00.161528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:00.161792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:00.168752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:00.170176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:57:00.182253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:00.183283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:00.184226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 627/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:57:00.189323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:00.306667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923681af-8f52-465a-b3b4-74bd99536184 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:00.344338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:00.344896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:00.345176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:00.345507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:00.346146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:00.346463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:00.347988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c92382-783e-4ece-80cc-bade196dfeaf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:00.366325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 618/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:00.367909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:00.499820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75861d-0964-4b43-926d-c0f1a33f6845 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:00.537891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:00.538518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:00.538795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:00.539153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:00.539888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] RBAC: Adding query filter params (role.id=3df158de7ee04605945a4d8e51c9b24e, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=bd2e18144a794d36b3d81cd3c8ef9cc3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:00.540191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:00.541621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db0617-0ada-4f9a-b176-c9e94126445d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:00.547747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 628/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:57:00 2022] GET /identity/v3/role_assignments?user.id=bd2e18144a794d36b3d81cd3c8ef9cc3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3df158de7ee04605945a4d8e51c9b24e => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:02.526264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70263f-9cc2-4f2a-a0c1-a2491f45a6fe None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:02.526851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70263f-9cc2-4f2a-a0c1-a2491f45a6fe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:02.526959 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70263f-9cc2-4f2a-a0c1-a2491f45a6fe None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:02.528369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 619/1247] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:02.536255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:02.536741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:02.536741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:02.543760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:02.558625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:02.559992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:02.636335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 629/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:02.937918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70263f-9cc2-4f2a-a0c1-a2491f45a6fe None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:02.957180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:03.045179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70263f-9cc2-4f2a-a0c1-a2491f45a6fe None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:03.083012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:03.083667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:03.083970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:03.084207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:03.097797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:57:03.103457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:03.104620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:03.105808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 620/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:57:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:03.110767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:03.222053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2f2e7-65f7-47e3-bd85-526d963f5bb9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:03.254392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:03.261768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:03.262397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:03.262674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:03.262945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:03.263658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:03.263959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:03.265343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:03.284556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 630/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:57:03.292654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:03.385637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:03.415998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c647ce2-ee5d-486e-bcf9-9afece1b0126 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:03.456784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:03.457405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:03.457683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:03.457946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:03.469328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:03.470837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:03.487494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:03.488614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:03.489749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:03.489749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:03.489749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:03.489749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 621/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 03:57:03.495118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:03.606115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bcb79-7981-4a77-9015-2a477698be71 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:03.645450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:03.646049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:03.646323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:03.646585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:03.647525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:03.647819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:03.649175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:03.664634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 631/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:03.670559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:03.780119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe2222-d1ec-4d02-a71b-7b5b13a76e71 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:03.808786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:03.817604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:03.818239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:03.818512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:03.818832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:03.826833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:03.828367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:57:03.840886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:03.842079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:03.843439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 622/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:03.848615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:03.910874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:03.962658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf851f9-cbcc-44a7-8095-862cf4cfd7e0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:04.005470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:04.006083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:04.006547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:04.007007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:04.007921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:04.008222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:04.009674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:04.029023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 632/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:57:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:04.035067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:04.143189 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f1f0d-d458-49fb-a4ef-615159409aae None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:04.178447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:04.179009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:04.179323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:04.179590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:04.240934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:04.255740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:04.257549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:04.376735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 623/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:57:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:57:04.382846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:04.485746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfb81e-16fe-4756-8857-a888a78e8760 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:04.501353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:04.525839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:04.526498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:04.526791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:04.527072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:04.538245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:04.539976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:04.554707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:04.555892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:04.557013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:04.557013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:04.557013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:04.557013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 633/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:04.561666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:04.656679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:04.670472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffea451-4b9f-4678-94da-73cced7e507f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:04.708671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:04.709550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:04.709902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:04.710195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:04.710882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:04.711240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:04.713680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:04.727214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 624/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:04.734131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:04.846818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0329-59a6-44ce-ac4d-f776b6b9954f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:04.890099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:04.890726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:04.891004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:04.891428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:04.899189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:04.900763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:57:04.913667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:04.914629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:04.915579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 634/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:57:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:04.921751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:05.029073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7825e-4a50-440e-9ab4-bd971df8c9b4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:05.065097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:05.065731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:05.066007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:05.066272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:05.067004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:05.067335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:05.068895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b9fc7-2e9c-4f68-ac3f-c5550df4fd34 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:05.082870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 625/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:57:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:05.090266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:05.119324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:05.196281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923f3bb-5105-4b7b-980d-f2588730f7f9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:05.231897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:05.232502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:05.232819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:05.233043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:05.233914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=bd2e18144a794d36b3d81cd3c8ef9cc3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:05.234218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:05.235780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54d226-69c4-4b53-8486-2a506e52ce20 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:05.242763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 635/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:57:05 2022] GET /identity/v3/role_assignments?user.id=bd2e18144a794d36b3d81cd3c8ef9cc3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:07.124230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6613-7491-4376-8eab-e47f40ca4784 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:07.125089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6613-7491-4376-8eab-e47f40ca4784 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:07.125089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b6613-7491-4376-8eab-e47f40ca4784 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:07.127365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 626/1261] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:07.132702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:07.133164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:07.133266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:07.152392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:07.153566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:07.237634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 636/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:07.502706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b6613-7491-4376-8eab-e47f40ca4784 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:07.520746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:07.618870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b6613-7491-4376-8eab-e47f40ca4784 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:07.664684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:07.665572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:07.665934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:07.666263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:07.681319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:57:07.686290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:07.687405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:07.688409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 627/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:07.696537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:07.722927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:07.816609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17b7db-b1b5-4ff0-af9d-b38844dc0968 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:07.853981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:07.854572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:07.854847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:07.855142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:07.855804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:07.856101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:07.857466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:07.865616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 637/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:07.871915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:07.943050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:07.976980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2d714-4ef3-424e-a8f1-ee16567d7059 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.023375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.024002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.024279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.024552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.034679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.036163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:08.050508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:08.051609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:08.052649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:08.052649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:08.052649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:08.052649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 628/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:08.058836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.151456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:08.172215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84daec-28ec-4044-b1ef-0a23c1815c27 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.207806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.208401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.208677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.208945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.209795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:08.210103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.211576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.223481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 638/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:08.229606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.337001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b633f9-a526-4d9d-af1a-e74ce18314f1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.356812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:08.376896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.377836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.378251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.378625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.392125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.394430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:57:08.406929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:08.407884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:08.408885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 629/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:08.417013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.533555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2e54f-2ec1-492d-9f71-f356369c13f9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.569108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.569711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.569983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.570250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.570966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:08.571293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.572702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.581257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:08.590416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 639/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:08.596407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.697851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7e1f6-708b-4629-bc61-8d4961817c7f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.735892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.736526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.736802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.737092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.749211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.750679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:08.766117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:08.767183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:08.768265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:08.768265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:08.768265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:08.768265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 630/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:08.774274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.878017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2850-d94e-44b7-9f05-3a7fb0976250 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:08.916531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:08.917124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:08.917402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:08.917672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:08.918309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:08.918606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:08.920000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:08.931674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 640/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:57:08.937635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:08.972769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:09.054008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d30076-5f87-49b6-9031-35a218b98285 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:09.091228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:09.091844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:09.092123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:09.092394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:09.099253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:09.100741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:57:09.112144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:09.113162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:09.114225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 631/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:09.119777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:09.183278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:09.239452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50ccf7-1852-4988-be13-15569ad1ade3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:09.275194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:09.275794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:09.276069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:09.276334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:09.277007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:09.277305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:09.278736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282b96b-3c90-402a-881e-19517be64622 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:09.291933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 641/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:09.298200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:09.402243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:09.412997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b8e4-ac2a-40b3-9ea4-e9a33b33acd3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:09.447813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:09.448458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:09.448733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:09.448987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] PATH_INFO: `/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/bd2e18144a794d36b3d81cd3c8ef9cc3/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:09.450918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d2fe1d7f6114b92a3075d8345d8a067, user_id=bd2e18144a794d36b3d81cd3c8ef9cc3, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:09.452393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc05f3-4f03-4ed8-b0f4-f9b1125fa091 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:09.472714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 632/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 31 03:57:09 2022] PUT /identity/v3/projects/6d2fe1d7f6114b92a3075d8345d8a067/users/bd2e18144a794d36b3d81cd3c8ef9cc3/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 03:57:11.345416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998feb0-7f46-449e-ac00-5c5834e17909 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:11.346236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998feb0-7f46-449e-ac00-5c5834e17909 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:11.346236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998feb0-7f46-449e-ac00-5c5834e17909 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:11.348553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 642/1274] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:11.354052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:11.354488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:11.354576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:11.375108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:11.376281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:11.451225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 633/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:11.714183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998feb0-7f46-449e-ac00-5c5834e17909 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:11.834296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998feb0-7f46-449e-ac00-5c5834e17909 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:11.851829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:11.874323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:11.875241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:11.875583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:11.875921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:11.893550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 03:57:11.899627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:11.900754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:11.902071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 643/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 31 03:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:11.907559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:11.996847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:12.017313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9b3e4-35db-401c-9b6e-01605d526c21 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.056703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.057312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.057586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.057846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:12.058504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:12.058799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:12.060211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:12.069235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 634/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 31 03:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:12.077853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:12.183375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad055cb3-b11a-4c11-a86f-fc04b9002ea0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.218105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.218711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.218991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.219283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:12.229796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:12.231272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:12.246580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:12.247713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:12.249016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:12.249016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:12.249016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:12.249016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 644/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:12.253648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:12.375387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8358f2-86af-4b54-bab8-d3c580461ca4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.415191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.415801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.416085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.416357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:12.417005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:12.417306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:12.418703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:12.431209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 635/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:12.437290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:12.470976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:12.551092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83a4aa-11c9-455a-8732-e73cd6c3282b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.594563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.595469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.595843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.596180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:12.605285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:12.606721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 03:57:12.619080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:12.620138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:12.621263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 645/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:12.628498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:12.632813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:12.740188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afcf22-c2f6-4bfd-a924-3291e9e44e4b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.776296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.776903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.777184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.777458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:12.778187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:12.778492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:12.779971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:12.797995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 636/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 31 03:57:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:12.804721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:12.913989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:12.926934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9212f-3ffe-4397-a389-fa40be19c1f0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:12.962813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:12.963439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:12.963717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:12.963983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:13.042763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:13.044248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:13.145937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:13.157208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 646/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 31 03:57:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 03:57:13.163510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:13.241823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:13.284812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba897d-efe9-4275-b718-9960b3c2c3ca None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:13.320589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:13.321216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:13.321488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:13.321752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:13.332121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:13.333573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 03:57:13.348443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:13.349791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 03:57:13.357020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 03:57:13.357020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 03:57:13.357020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:13.357020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 637/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 03:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:13.358189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:13.465847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79fab-78bf-49be-86e4-cadce4a375a5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:13.502017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:13.502610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:13.502887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:13.503177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:13.503812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:13.504111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:13.505459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:13.516653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 647/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 31 03:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:13.523562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:13.635145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5689a8-4bfb-4235-bc01-801fd8758fdb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:13.649095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:13.671230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:13.671865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:13.672143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:13.672416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:13.679571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:13.681028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 03:57:13.692593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:13.693602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Aug 31 03:57:13.694863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 638/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 31 03:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:13.702160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:13.759398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:13.811351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df2bb6-f302-4058-b6cd-729d759244ac None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:13.846595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:13.847220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:13.847493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:13.847765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:13.848450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:13.848751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:13.850367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94df1a-4b37-4e05-85a5-d394c89f5f93 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:13.862277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 648/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 31 03:57:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:13.869766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:13.982566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e2cf4-28d5-47ae-aa32-11957c25f249 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:14.017663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:14.018199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:14.018467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:14.018734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:14.019479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] RBAC: Adding query filter params (role.id=3f9414c863e34276b7483d736ee6a5b0, scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067, user.id=bd2e18144a794d36b3d81cd3c8ef9cc3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:14.019772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:14.021200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1bfff-c737-476b-ad00-4510e3ef89e1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:14.027175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 639/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 31 03:57:13 2022] GET /identity/v3/role_assignments?user.id=bd2e18144a794d36b3d81cd3c8ef9cc3&scope.project.id=6d2fe1d7f6114b92a3075d8345d8a067&role.id=3f9414c863e34276b7483d736ee6a5b0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:15.901323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ec98-43eb-4888-94bd-b401c3992ad6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:15.901929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ec98-43eb-4888-94bd-b401c3992ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:15.901929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ec98-43eb-4888-94bd-b401c3992ad6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:15.903341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 649/1288] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:15.909660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:15.910462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:15.910634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:15.936435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:15.937616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:15.970303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:16.013439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 640/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:16.289609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7ec98-43eb-4888-94bd-b401c3992ad6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:16.398103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7ec98-43eb-4888-94bd-b401c3992ad6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:16.409251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:16.434574 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:16.435218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:16.435495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:16.435762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:16.436333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:16.437685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:16.446986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:16.448273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:16.449364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 650/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:57:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:16.454390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:16.558573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2e69a-b319-4259-9f07-0676e5a33ab6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:16.600159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:16.600943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:16.601278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:16.601650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:16.602540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:16.602962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:16.605168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830d971-8985-47e9-86e5-1e63beb15d5f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:16.613818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 641/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 31 03:57:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 03:57:16.620594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:16.731505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a982-ae9a-4d1a-a1ca-75a86d0c54b0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:16.777935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:16.778862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:16.779390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:16.779680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:16.780499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:16.781151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:16.783101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3759-2288-401a-bb01-24ffc27c6997 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:16.789476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 651/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 31 03:57:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:18.660603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2289-7d69-4e49-acb2-9c69a98e13b5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:18.661169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2289-7d69-4e49-acb2-9c69a98e13b5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:18.661169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2289-7d69-4e49-acb2-9c69a98e13b5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:18.662396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 642/1293] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:18.667613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f3a6-efb4-47fa-b1ed-4c35fdecf6dd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:18.668079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f3a6-efb4-47fa-b1ed-4c35fdecf6dd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:18.668179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f3a6-efb4-47fa-b1ed-4c35fdecf6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:18.688007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5f75f3a6-efb4-47fa-b1ed-4c35fdecf6dd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:18.689186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f75f3a6-efb4-47fa-b1ed-4c35fdecf6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:18.724911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:18.772950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 652/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:19.047340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec2289-7d69-4e49-acb2-9c69a98e13b5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:19.088726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:19.165801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec2289-7d69-4e49-acb2-9c69a98e13b5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:19.201102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:19.201699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:19.201975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:19.202243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:19.202889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:19.204294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1ae2f-102a-41b2-b56e-717a27c7441b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:19.221328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 643/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 31 03:57:19 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:57:21.095863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b28-2dc7-4d01-85f0-71bc4ea93497 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:21.096797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b28-2dc7-4d01-85f0-71bc4ea93497 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:21.096797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b28-2dc7-4d01-85f0-71bc4ea93497 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:21.098844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 653/1296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:21.104396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:21.104825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:21.104934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:21.124622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:21.125789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:21.200039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 644/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:21.463594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0b28-2dc7-4d01-85f0-71bc4ea93497 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:21.576968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0b28-2dc7-4d01-85f0-71bc4ea93497 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:21.612218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:21.612826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:21.613101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:21.613367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:21.613943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:21.615382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:21.624591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:21.625719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:21.626769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 654/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:57:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:21.634057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:21.658174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:21.745925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9c12c-16b7-45b4-bca8-97ac5ecf154a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:21.782698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:21.783394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:21.783671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:21.784004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:21.784725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] RBAC: Adding query filter params (name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:21.785030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:21.786638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:21.792932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 645/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 31 03:57:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:21.800296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:21.833518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:21.918412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac60e5-b6ba-4782-8c51-41198493abf3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:21.954047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:21.954636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:21.954912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:21.955211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:21.955913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] RBAC: Adding query filter params (interface=public, service_id=9253f09ba11b44dd979b05278b3b8be2, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:21.956214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:21.957583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647e14-4163-4fa4-8334-6500f6e8dd16 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:21.963214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 655/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 31 03:57:21 2022] GET /identity/v3/endpoints?service_id=9253f09ba11b44dd979b05278b3b8be2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:21.969416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:22.057459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:22.077291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa34020-c742-40d9-906b-91cfbb6da7ba None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:22.122174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:22.122932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:22.123190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:22.123459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:22.124065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:22.125447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2d10-253d-4582-8f77-1a99f9b37b76 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:22.133617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 646/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 31 03:57:21 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 03:57:23.976961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc064723-1fbf-43ef-9461-3252d20d20a9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:23.977874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc064723-1fbf-43ef-9461-3252d20d20a9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:23.977874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc064723-1fbf-43ef-9461-3252d20d20a9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:23.980047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 656/1302] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:57:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:57:23.985502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500630-9697-4294-9e22-de01bfc78036 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:23.985936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500630-9697-4294-9e22-de01bfc78036 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:23.986038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500630-9697-4294-9e22-de01bfc78036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:24.006162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-14500630-9697-4294-9e22-de01bfc78036 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:57:24.007355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-14500630-9697-4294-9e22-de01bfc78036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:57:24.076150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 647/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:57:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 03:57:24.336367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc064723-1fbf-43ef-9461-3252d20d20a9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:24.447370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:24.459423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc064723-1fbf-43ef-9461-3252d20d20a9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:24.500691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:24.501328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:24.501605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:24.501845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:24.502440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:24.503824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 03:57:24.513285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 03:57:24.514470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Aug 31 03:57:24.515396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 657/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 31 03:57:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 03:57:24.521350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14500630-9697-4294-9e22-de01bfc78036 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:24.602600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:24.627693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14500630-9697-4294-9e22-de01bfc78036 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:24.662898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:24.663513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:24.663786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:24.664055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:24.664670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 03:57:24.664962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:24.666338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482807-2cfa-452e-b098-b0c8171cd4e3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:24.671661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 648/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 31 03:57:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 03:57:24.677787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:57:24.790902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef83dab-62cf-4fbb-820f-1caebd368fe9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:57:24.825619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:57:24.826230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:57:24.826502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:57:24.826762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:57:24.827428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:57:24.828777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f88030-2d42-42d2-9bf3-fb316b00a632 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:57:24.851705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 658/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 03:57:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 03:57:34.668570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=30 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:57:34.846560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:28.669110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b84945-45fb-4b40-9e6d-69188feead2a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:28.669598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b84945-45fb-4b40-9e6d-69188feead2a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:28.669677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b84945-45fb-4b40-9e6d-69188feead2a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:28.675572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 649/1307] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:28.676413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6444-9ced-4c53-8161-f333b9d82cc9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:28.676948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6444-9ced-4c53-8161-f333b9d82cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:28.676948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6444-9ced-4c53-8161-f333b9d82cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:28.818369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bf4e6444-9ced-4c53-8161-f333b9d82cc9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:28.819580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bf4e6444-9ced-4c53-8161-f333b9d82cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:28.948598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 659/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:29.241864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:29.242143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:29.242421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:29.244105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 650/1309] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 31 03:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:29.250719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a432a-6fc9-42ee-849a-f4a71b69df5f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:29.251011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a432a-6fc9-42ee-849a-f4a71b69df5f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:29.251333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a432a-6fc9-42ee-849a-f4a71b69df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:29.338921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-130a432a-6fc9-42ee-849a-f4a71b69df5f None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:29.340175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-130a432a-6fc9-42ee-849a-f4a71b69df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:29.365347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:29.425640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 660/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 31 03:58:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 03:58:29.434875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:29.437068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:29.554286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29df08-6dd0-4b69-9052-1050e6c22beb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:29.633402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:29.634427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:29.634984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:29.635483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:29.637308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:29.697568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:29.758604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:29.760447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0468f8-d849-48df-8a8b-dd87d7489bd9 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:29.782873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 651/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:32.017125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:32.017692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:32.017692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:32.019057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 661/1312] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:32.024352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87390de-0401-4da5-97a7-a06fd5ebb570 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:32.024796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87390de-0401-4da5-97a7-a06fd5ebb570 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:32.024888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87390de-0401-4da5-97a7-a06fd5ebb570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:32.101029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b87390de-0401-4da5-97a7-a06fd5ebb570 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:32.102247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b87390de-0401-4da5-97a7-a06fd5ebb570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:32.207192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 652/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:32.489287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:32.491232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:32.597218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9c6a8-cca2-49e3-9ed2-05a29be3edc5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:32.674654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:32.677307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:32.677919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:32.678201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:32.678478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:32.680766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:32.803254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:32.804652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cdcb7-6d62-4e2a-b546-680006ce705b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:32.823482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 662/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 03:58:35.039692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:35.040602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:35.040602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:35.042762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 653/1315] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:35.048252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acbc6d-fd22-4a89-8751-d39b0293b89f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:35.048720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acbc6d-fd22-4a89-8751-d39b0293b89f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:35.049077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acbc6d-fd22-4a89-8751-d39b0293b89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:35.121093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-99acbc6d-fd22-4a89-8751-d39b0293b89f None None] MFA Rules not processed for user `ca8683acefdd46fa801a7427335c1529`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:35.122281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-99acbc6d-fd22-4a89-8751-d39b0293b89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:35.181633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:35.230299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 663/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:35.515238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:35.517807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:35.533898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:35.646278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def460ae-9e54-4ad3-98ce-6e829decc74b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:35.737637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:35.738245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:35.738518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:35.738782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:35.740748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:35.807080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:35.857562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:35.858999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d68bdf-a53a-415c-83b3-67045c71969f service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:35.879513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 654/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:38.372327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:38.372852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:38.372852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:38.374165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 664/1318] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:38.380883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afb14-c4cb-4413-bdb1-1ae540911ea9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:38.381634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afb14-c4cb-4413-bdb1-1ae540911ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:38.381762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624afb14-c4cb-4413-bdb1-1ae540911ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:38.471638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-624afb14-c4cb-4413-bdb1-1ae540911ea9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:38.472899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-624afb14-c4cb-4413-bdb1-1ae540911ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:38.580896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 655/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:38.877293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:38.879794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:38.994252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b7c64-438b-4d13-8946-1b2dca8b6695 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:39.071195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:39.071795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:39.072070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:39.072343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:39.073926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:39.082311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:39.204798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:39.206235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b00e-8621-4674-942c-ef519fc743e0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:39.224762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 665/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:41.818469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b8c2-158b-45a3-9882-2d44ae2cee78 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:41.819012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b8c2-158b-45a3-9882-2d44ae2cee78 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:41.819012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b8c2-158b-45a3-9882-2d44ae2cee78 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:41.820494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 656/1321] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:41.826062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318602a2-3d0d-4c4e-8a18-94623a68ca5d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:41.826519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318602a2-3d0d-4c4e-8a18-94623a68ca5d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:41.826606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318602a2-3d0d-4c4e-8a18-94623a68ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:41.893355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-318602a2-3d0d-4c4e-8a18-94623a68ca5d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:41.894553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-318602a2-3d0d-4c4e-8a18-94623a68ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:41.952735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:41.993796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 666/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:42.286816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:42.287205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:42.287476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:42.288836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 657/1323] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 31 03:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:42.295006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c08a2-80a1-4002-be38-a06185b538bf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:42.295392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c08a2-80a1-4002-be38-a06185b538bf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:42.295669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c08a2-80a1-4002-be38-a06185b538bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:42.375667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7c1c08a2-80a1-4002-be38-a06185b538bf None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:42.377409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7c1c08a2-80a1-4002-be38-a06185b538bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:42.457657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 667/1324] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 31 03:58:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:42.466644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:42.468244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:42.489974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:42.579194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cfe37-6f42-4016-9571-6e6a31d51c81 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:42.659966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:42.660482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:42.660760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:42.661033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:42.662669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:42.739021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:42.778969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:42.780447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332b71-be78-42f2-aeae-3be52cd5d376 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:42.799568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 658/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 03:58:44.972883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:44.973471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:44.973471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:44.974764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 668/1326] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:44.980080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a4f91-17ba-4377-932e-2de412d6c491 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:44.980560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a4f91-17ba-4377-932e-2de412d6c491 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:44.980654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a4f91-17ba-4377-932e-2de412d6c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:45.050732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-225a4f91-17ba-4377-932e-2de412d6c491 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:45.060486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-225a4f91-17ba-4377-932e-2de412d6c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:45.173442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 659/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:45.449512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:45.451057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:45.516146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:45.559904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2b95c-546f-4661-aa75-f11a360ae044 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:45.632809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:45.633413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:45.633697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:45.633965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:45.635594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:45.745773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:45.748151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:45.749890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce8c7-a9ca-43c7-be6e-cb0393d63e4e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:45.769911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 669/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:47.712956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:47.713549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:47.713549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:47.714856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 660/1329] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:47.720615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820479-122e-40c9-9c59-712eb29f6c80 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:47.721126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820479-122e-40c9-9c59-712eb29f6c80 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:47.721228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820479-122e-40c9-9c59-712eb29f6c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:47.794656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c7820479-122e-40c9-9c59-712eb29f6c80 None None] MFA Rules not processed for user `ca8683acefdd46fa801a7427335c1529`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:47.795957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c7820479-122e-40c9-9c59-712eb29f6c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:47.902198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 670/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:48.181117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:48.182678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:48.218229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:48.308172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0fa0a-360a-491b-a552-5f42f2853718 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:48.395869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:48.396484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:48.396740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:48.397057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:48.398671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:48.491633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:48.524489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:48.525935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20016-c6bd-482c-934f-7b1d6f54f048 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:48.544365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 661/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:50.638764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:50.639705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:50.639705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:50.641991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 671/1332] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:50.650717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935baad-8dea-4864-b628-eadc172abbc5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:50.651238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935baad-8dea-4864-b628-eadc172abbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:50.651402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935baad-8dea-4864-b628-eadc172abbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:50.722689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4935baad-8dea-4864-b628-eadc172abbc5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:50.723946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4935baad-8dea-4864-b628-eadc172abbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:50.837079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 662/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:51.119995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:51.121561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:51.198709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:51.236564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0528f8-8142-4688-9624-3d912d171368 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:51.314484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:51.315062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:51.315374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:51.315637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:51.317215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:51.430465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:51.431914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b48bdd-0572-4bbb-be8a-2350c96e8174 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:51.442363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:51.450771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 672/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:53.433809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:53.434685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:53.434685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:53.436944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 663/1335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:53.443175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b3-cee0-439e-9ff9-30f336e63de1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:53.443940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b3-cee0-439e-9ff9-30f336e63de1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:53.444083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b3-cee0-439e-9ff9-30f336e63de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:53.517049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8338d7b3-cee0-439e-9ff9-30f336e63de1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:53.518272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8338d7b3-cee0-439e-9ff9-30f336e63de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:53.619975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 673/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:53.915183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:53.917704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:53.964875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:54.041450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15acd76-2f8f-4f82-b041-d8508bed3a91 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:54.119371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:54.119980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:54.120255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:54.120529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:54.122125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:54.217301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:54.244776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:54.246232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b832128-e778-44ea-bcce-0b82d4f5ce7d service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:54.264734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 664/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:58:56.484171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:56.485241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:56.485241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:56.486411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 674/1338] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:58:56.492212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87d05-e968-4220-b0ce-59c0d015f656 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:56.492636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87d05-e968-4220-b0ce-59c0d015f656 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:56.492727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f87d05-e968-4220-b0ce-59c0d015f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:56.563276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-95f87d05-e968-4220-b0ce-59c0d015f656 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:58:56.564517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-95f87d05-e968-4220-b0ce-59c0d015f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:56.670681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 665/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:58:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:58:56.965330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:58:56.967899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:57.056496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:58:57.086148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc8790-cee6-4b6f-92b7-df1f81684260 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:58:57.171843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:58:57.172527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:58:57.172869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:58:57.173085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:58:57.174686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:58:57.292557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:58:57.294001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c59e62-d4cc-47cf-afad-77e652ab8fc4 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:58:57.318718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 675/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:58:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:01.218823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:01.219913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:01.219913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:01.220792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 666/1341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:01.226294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ef4b-f6e6-4c56-b341-bdb5bb0f5d8a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:01.226741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ef4b-f6e6-4c56-b341-bdb5bb0f5d8a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:01.226821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404ef4b-f6e6-4c56-b341-bdb5bb0f5d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:01.232395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:01.300729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f404ef4b-f6e6-4c56-b341-bdb5bb0f5d8a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:01.301943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f404ef4b-f6e6-4c56-b341-bdb5bb0f5d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:01.400709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 676/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:01.678619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:01.680188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:01.735935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:01.796599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c341-fd26-459e-855c-9c01ced488f0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:01.872688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:01.873293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:01.873567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:01.873829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:01.875566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:01.971258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:01.988295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:01.990089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307e165-e313-424a-a8b2-1f5489d4936e service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:02.008602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 667/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:04.882657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:04.883424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:04.883424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:04.884457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 677/1344] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:04.889901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f3bed-33dc-4044-8d62-0d0db697f816 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:04.890360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f3bed-33dc-4044-8d62-0d0db697f816 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:04.890511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f3bed-33dc-4044-8d62-0d0db697f816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:04.957390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d35f3bed-33dc-4044-8d62-0d0db697f816 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:04.958726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d35f3bed-33dc-4044-8d62-0d0db697f816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:05.064082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 668/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:05.347060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:05.348626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:05.446373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:05.465174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0ca3-fd2a-43e6-b38d-efc58c04c9e5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:05.542732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:05.543284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:05.543558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:05.543824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:05.545399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:05.658630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:05.660115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83588b7d-1c2a-4c55-a616-9ebe2196adb5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:05.678486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 678/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:09.676743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:09.677331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:09.677331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:09.678653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 669/1347] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:09.684613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8782ea5-3641-4276-9333-edfa6c98a625 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:09.685122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8782ea5-3641-4276-9333-edfa6c98a625 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:09.685122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8782ea5-3641-4276-9333-edfa6c98a625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:09.702529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:09.763705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d8782ea5-3641-4276-9333-edfa6c98a625 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:09.764928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d8782ea5-3641-4276-9333-edfa6c98a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:09.866611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 679/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:10.147368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:10.148913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:10.212607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:10.260492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b3e24-690f-497a-aeb0-2b728e73c158 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:10.339423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:10.340034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:10.340313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:10.340581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:10.342190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:10.464230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:10.469714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:10.471205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1fb75-683e-4a7a-8b85-10eef261f5f6 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:10.490892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 670/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:15.338967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87635751-5545-463b-8792-3bb55e126ff3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:15.339412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87635751-5545-463b-8792-3bb55e126ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:15.339545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87635751-5545-463b-8792-3bb55e126ff3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:15.341854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 680/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 03:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:15.351174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafa57-8587-468c-be38-fa8bf87178f1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:15.351174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafa57-8587-468c-be38-fa8bf87178f1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:15.351174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acafa57-8587-468c-be38-fa8bf87178f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:15.451300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9acafa57-8587-468c-be38-fa8bf87178f1 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:15.453597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9acafa57-8587-468c-be38-fa8bf87178f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:15.612520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 671/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 03:59:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:19.840665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a273f-554b-4014-bee3-ca373bea8cd2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:19.841647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a273f-554b-4014-bee3-ca373bea8cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:19.841927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a273f-554b-4014-bee3-ca373bea8cd2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:19.843893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 681/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 03:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:59:19.849128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88521cd9-45e7-43ae-92c1-3a8d3dbdfa2d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:19.850011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88521cd9-45e7-43ae-92c1-3a8d3dbdfa2d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:19.850539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88521cd9-45e7-43ae-92c1-3a8d3dbdfa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:19.864510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=59 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:19.913796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:19.946295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-88521cd9-45e7-43ae-92c1-3a8d3dbdfa2d None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:19.948656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-88521cd9-45e7-43ae-92c1-3a8d3dbdfa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:20.075134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 672/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 03:59:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:21.148139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac325e9-e7c3-4c15-a4cd-be933e859d9c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:21.148728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac325e9-e7c3-4c15-a4cd-be933e859d9c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:21.148728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac325e9-e7c3-4c15-a4cd-be933e859d9c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:21.150189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 682/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 03:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:21.155144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a5b5-5e04-4b17-862f-caac1dbef9e6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:21.155606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a5b5-5e04-4b17-862f-caac1dbef9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:21.155773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825a5b5-5e04-4b17-862f-caac1dbef9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:21.236994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e825a5b5-5e04-4b17-862f-caac1dbef9e6 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:21.238510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e825a5b5-5e04-4b17-862f-caac1dbef9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:21.283747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:21.352455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 673/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 03:59:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:24.801041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a4c3-2eeb-473a-be45-2dfc894f78fd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:24.801041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a4c3-2eeb-473a-be45-2dfc894f78fd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:24.801635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0a4c3-2eeb-473a-be45-2dfc894f78fd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:24.802464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 683/1356] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:59:24.809219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03570c-ab74-4c02-b590-eeb7568d0c36 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:24.809749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03570c-ab74-4c02-b590-eeb7568d0c36 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:24.809968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03570c-ab74-4c02-b590-eeb7568d0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:24.900326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6f03570c-ab74-4c02-b590-eeb7568d0c36 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:24.901573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6f03570c-ab74-4c02-b590-eeb7568d0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:25.018544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 674/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:25.319853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:25.320142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:25.320395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:25.321707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 684/1358] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 31 03:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:25.332035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9f39-180c-41e7-a704-4fbee8ac23e8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:25.332327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9f39-180c-41e7-a704-4fbee8ac23e8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:25.332608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab9f39-180c-41e7-a704-4fbee8ac23e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:25.355451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:25.424031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-84ab9f39-180c-41e7-a704-4fbee8ac23e8 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:25.425235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-84ab9f39-180c-41e7-a704-4fbee8ac23e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:25.552874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 675/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 31 03:59:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:25.560448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:25.562012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:25.752914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59194cda-a3aa-47fc-bcbb-9fd6ee86c072 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:25.829235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:25.871780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:25.872381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:25.872658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:25.872899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:25.874614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:25.965338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:25.966941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9c01d-c600-4566-87f6-1ceed2d19fb7 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:25.985822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 685/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:28.080125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:28.080694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:28.080940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:28.082408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 676/1361] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:28.087778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf0f9-9a96-4399-8f9e-82748f7c7418 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:28.088210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf0f9-9a96-4399-8f9e-82748f7c7418 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:28.088311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf0f9-9a96-4399-8f9e-82748f7c7418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:28.160445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f4bbf0f9-9a96-4399-8f9e-82748f7c7418 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:28.161628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f4bbf0f9-9a96-4399-8f9e-82748f7c7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:28.207235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:28.291139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 686/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:28.589981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:28.591631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:28.660283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:28.751719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0289-a2bd-4b1e-8c80-81df1444bebe None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:28.895784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:28.896485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:28.896625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:28.897247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:28.898820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:28.923755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:28.997746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:28.999546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e2dd3-80f4-4b18-b1f1-f3cfe8d05cd6 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:29.018749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 677/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:31.284577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b04a-a3eb-4b4b-9e02-209ed3f7f4f5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:31.285587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b04a-a3eb-4b4b-9e02-209ed3f7f4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:31.286249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b04a-a3eb-4b4b-9e02-209ed3f7f4f5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:31.287836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 687/1364] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:31.293337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367efa7-6eac-4f28-af48-8aeeb08513a6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:31.293749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367efa7-6eac-4f28-af48-8aeeb08513a6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:31.293935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367efa7-6eac-4f28-af48-8aeeb08513a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:31.365225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5367efa7-6eac-4f28-af48-8aeeb08513a6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:31.366445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5367efa7-6eac-4f28-af48-8aeeb08513a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:31.466577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:31.469259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 678/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:31.757879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:31.758185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:31.758434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:31.760454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 688/1366] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 31 03:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:31.765474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745af93-11de-4a35-aba6-e68afcb3f6a7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:31.765949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745af93-11de-4a35-aba6-e68afcb3f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:31.766219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745af93-11de-4a35-aba6-e68afcb3f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:31.845551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b745af93-11de-4a35-aba6-e68afcb3f6a7 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:31.846714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b745af93-11de-4a35-aba6-e68afcb3f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:31.973844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 679/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 31 03:59:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:31.982057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:31.983700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:32.141170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2504c-fec4-497e-ac63-5cc624cdad04 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:32.162499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:32.290571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65712a-7feb-423f-a58c-146573accada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:32.292200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65712a-7feb-423f-a58c-146573accada service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:32.292845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65712a-7feb-423f-a58c-146573accada service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:32.293279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65712a-7feb-423f-a58c-146573accada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:32.295064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ba65712a-7feb-423f-a58c-146573accada service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:32.388251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65712a-7feb-423f-a58c-146573accada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:32.390343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65712a-7feb-423f-a58c-146573accada service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:32.410768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 689/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:32.789228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ea7fc-7df9-4cf7-b0d2-b53f6c129fae None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:32.789811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ea7fc-7df9-4cf7-b0d2-b53f6c129fae None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:32.790271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ea7fc-7df9-4cf7-b0d2-b53f6c129fae None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:32.792227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 680/1369] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:32.797837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d16a95-13f9-44cc-bdb3-734d98d5e5c5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:32.798107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d16a95-13f9-44cc-bdb3-734d98d5e5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:32.798387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d16a95-13f9-44cc-bdb3-734d98d5e5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:32.832051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:32.870775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d7d16a95-13f9-44cc-bdb3-734d98d5e5c5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:32.872002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7d16a95-13f9-44cc-bdb3-734d98d5e5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:32.976597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 690/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:33.547061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:33.547319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:33.547616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:33.549196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 681/1371] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 31 03:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 03:59:33.554075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479cd9c-8ede-4f71-b2cc-2635ada3e11e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:33.554636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479cd9c-8ede-4f71-b2cc-2635ada3e11e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:33.554636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479cd9c-8ede-4f71-b2cc-2635ada3e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:33.637868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d479cd9c-8ede-4f71-b2cc-2635ada3e11e None None] MFA Rules not processed for user `9a1ecf64ce544ee5aa7839acde57bea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:33.639232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d479cd9c-8ede-4f71-b2cc-2635ada3e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:33.694233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:33.725209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 691/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 31 03:59:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:33.733630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:33.735832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:33.792626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:33.848207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e49f1-67c4-43d3-9b6e-fe18d760de21 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:33.929257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:33.929866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:33.930175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:33.930463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:33.932028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:34.036750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:34.051633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:34.053230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06901188-c3c4-4482-80ff-a996be58d1eb service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:34.075761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 682/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:34.420332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:34.420541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:34.420661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:34.422022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 692/1374] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:34.427610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bc5b-82f0-41bb-afeb-e396b861b3e4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:34.427896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bc5b-82f0-41bb-afeb-e396b861b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:34.428170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87bc5b-82f0-41bb-afeb-e396b861b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:34.495513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-df87bc5b-82f0-41bb-afeb-e396b861b3e4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:34.496731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-df87bc5b-82f0-41bb-afeb-e396b861b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:34.594317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 683/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:34.883585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:34.885195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:35.033694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a77c7-64cc-4802-8214-2fdd269745bd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:35.086449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:35.151757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:35.152363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:35.152628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:35.152899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:35.154531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:35.245895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:35.247613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b50d3-943b-4cf7-a0a9-d6ec289700da service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:35.268021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 693/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:37.198651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:37.199204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:37.199204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:37.200571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 684/1377] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:37.205773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ae2c7-8bd1-4acf-ad84-939b605a45cf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:37.206205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ae2c7-8bd1-4acf-ad84-939b605a45cf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:37.206295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ae2c7-8bd1-4acf-ad84-939b605a45cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:37.276490 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-244ae2c7-8bd1-4acf-ad84-939b605a45cf None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:37.277649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-244ae2c7-8bd1-4acf-ad84-939b605a45cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:37.288388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:37.380330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 694/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:37.678152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:37.679712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:37.747739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:37.840259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc111c-bc88-42de-b3c1-d57515fd5049 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:37.958926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:37.959558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:37.959835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:37.960090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:37.961700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:37.981601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:38.058412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:38.060070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5d639-0b4b-4e30-b32d-7f10d172e2c3 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:38.080818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 685/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:40.086302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:40.086865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:40.086865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:40.088355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 695/1380] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:40.093711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c11f-34dc-4c04-862f-d1df0e5fda31 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:40.094699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c11f-34dc-4c04-862f-d1df0e5fda31 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:40.094976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c11f-34dc-4c04-862f-d1df0e5fda31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:40.169929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8d26c11f-34dc-4c04-862f-d1df0e5fda31 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:40.171526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8d26c11f-34dc-4c04-862f-d1df0e5fda31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:40.271814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:40.279209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 686/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:40.574458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:40.576044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:40.724245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:40.737850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170fab1-f3b6-4f25-8f69-0a12dbc5aa44 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:40.865342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:40.865950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:40.866231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:40.866505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:40.868141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:40.969842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:40.971484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e627d-c99b-4141-8b84-998c43de569a service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:40.983090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:40.993312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 696/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:43.131227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:43.131803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:43.131803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:43.133162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 687/1383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:43.139513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5186-d0d1-40d6-82e2-51ba5ccdc90d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:43.139960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5186-d0d1-40d6-82e2-51ba5ccdc90d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:43.140101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d5186-d0d1-40d6-82e2-51ba5ccdc90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:43.216692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-109d5186-d0d1-40d6-82e2-51ba5ccdc90d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:43.217961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-109d5186-d0d1-40d6-82e2-51ba5ccdc90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:43.319556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 697/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 03:59:43.599342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:43.600865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:43.753394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86a92-950b-4a4c-ae7e-b85648a8da0a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:43.834719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:43.884956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:43.885462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:43.885741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:43.886014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:43.887715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:44.000022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:44.001678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05b115-7b5a-46aa-be57-95502ec4364f service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:44.020425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 688/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:45.916812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:45.917580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:45.917580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:45.919438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 698/1386] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:45.925998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f103b9-41ac-4dfe-87b1-681c813dbaa6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:45.926460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f103b9-41ac-4dfe-87b1-681c813dbaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:45.926553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f103b9-41ac-4dfe-87b1-681c813dbaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:46.002359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-28f103b9-41ac-4dfe-87b1-681c813dbaa6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:46.003597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-28f103b9-41ac-4dfe-87b1-681c813dbaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:46.037925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:46.116951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 689/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:46.400584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:46.402628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:46.481613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:46.572986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0750e-907e-4084-8d89-5c1e9f1f39b9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:46.703955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:46.704540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:46.704852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:46.705117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:46.706736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:46.751416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:46.806646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:46.808741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a5d8-ebba-4376-b132-aa747882877f service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:46.832829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 699/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:49.272590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:49.273176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:49.273321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:49.274847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 690/1389] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:49.281196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0924-abdb-4b11-b0ac-46ac6f0773c4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:49.281664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0924-abdb-4b11-b0ac-46ac6f0773c4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:49.281771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0924-abdb-4b11-b0ac-46ac6f0773c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:49.351008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-acda0924-abdb-4b11-b0ac-46ac6f0773c4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:49.352232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-acda0924-abdb-4b11-b0ac-46ac6f0773c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:49.452711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 700/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:49.734953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:49.736875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:49.887578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e62601-73a3-42a6-9701-e21cc0c930c2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:49.905036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:50.003623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:50.004143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:50.004442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:50.004712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:50.006331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:50.096357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:50.097958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c27b-7ac0-4f57-886e-fb413e61d336 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:50.117064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 691/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:51.164923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1525a7-124a-445b-aafa-368b4b237cc3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:51.165508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1525a7-124a-445b-aafa-368b4b237cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:51.165508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1525a7-124a-445b-aafa-368b4b237cc3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:51.166857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 701/1392] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:51.172514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622c38-397b-421b-b5a4-8cf5dab814a9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:51.172960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622c38-397b-421b-b5a4-8cf5dab814a9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:51.173062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91622c38-397b-421b-b5a4-8cf5dab814a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:51.199038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:51.245400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-91622c38-397b-421b-b5a4-8cf5dab814a9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:51.246628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-91622c38-397b-421b-b5a4-8cf5dab814a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:51.345978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 692/1393] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:51.648296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:51.648576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:51.648847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:51.650353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 702/1394] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 31 03:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:51.655620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e329-2e51-4eae-8530-ab92a4cee1af None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:51.655987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e329-2e51-4eae-8530-ab92a4cee1af None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:51.656331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4e329-2e51-4eae-8530-ab92a4cee1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:51.735280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6ac4e329-2e51-4eae-8530-ab92a4cee1af None None] MFA Rules not processed for user `a77972e7e64f47c7a3fe7153841976c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:51.736489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6ac4e329-2e51-4eae-8530-ab92a4cee1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:51.778912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:51.815764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 693/1395] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 31 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:51.823583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:51.825225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:51.839452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:51.934785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1df49-49e5-48b3-96e1-8b81b81ec0b1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:52.010896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:52.011528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:52.011805 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:52.012097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:52.013732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:52.050173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2526d0-3a0b-4360-8b5e-2624995b89de None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:52.050488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2526d0-3a0b-4360-8b5e-2624995b89de None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:52.050759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2526d0-3a0b-4360-8b5e-2624995b89de None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:52.052133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 694/1396] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:52.056989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4207f0-cc63-4315-bc5a-0090226f03e8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:52.057275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4207f0-cc63-4315-bc5a-0090226f03e8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:52.057554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4207f0-cc63-4315-bc5a-0090226f03e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:52.083729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:52.126919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:52.128199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2f4207f0-cc63-4315-bc5a-0090226f03e8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:52.128403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:52.129414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2f4207f0-cc63-4315-bc5a-0090226f03e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:52.147151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 703/1397] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:52.228322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 695/1398] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 03:59:52.521628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:52.522958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:52.629565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06fce4-4004-48db-92aa-f6ebede0b058 service glance] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:52.698758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:52.755368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:52.755995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:52.756278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:52.756553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:52.758318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:52.861123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:52.863405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e744f3-764a-4de0-a522-78d53614f5da service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:52.892717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 704/1399] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:52.913164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502e43-1ef1-4991-a2eb-eca1d23def43 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:52.913462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502e43-1ef1-4991-a2eb-eca1d23def43 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:52.913724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65502e43-1ef1-4991-a2eb-eca1d23def43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:52.991273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:53.005247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-65502e43-1ef1-4991-a2eb-eca1d23def43 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:53.006520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-65502e43-1ef1-4991-a2eb-eca1d23def43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:53.128693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 696/1400] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:53.178595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:53.178890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:53.179215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:53.180587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 705/1401] 10.222.0.18 () {54 vars in 943 bytes} [Wed Aug 31 03:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:53.187847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b52e-82f1-4116-85fc-72644018f3e4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:53.188111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b52e-82f1-4116-85fc-72644018f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:53.188401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b52e-82f1-4116-85fc-72644018f3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:53.288158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7b05b52e-82f1-4116-85fc-72644018f3e4 None None] MFA Rules not processed for user `b526cee1dc234083a7f4897fee8e22b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:53.289419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b05b52e-82f1-4116-85fc-72644018f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:53.372284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:53.415888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 697/1402] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Aug 31 03:59:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:53.425779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:53.427350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:53.528103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:53.599601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250beb0c-e47d-4ec8-8f20-835273c5f719 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:53.729686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:53.730711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:53.731143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:53.731420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:53.734061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:53.811196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:53.830915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:53.832566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319668a-5bc0-443a-a688-acd2214d1f2d service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:53.840860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 706/1403] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 03:59:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 03:59:54.913823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:54.916175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:54.916175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:54.916175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 698/1404] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:54.921742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c45aa-6af6-47a5-9586-2bb5e51ce076 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:54.922204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c45aa-6af6-47a5-9586-2bb5e51ce076 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:54.922312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c45aa-6af6-47a5-9586-2bb5e51ce076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:54.994704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f03c45aa-6af6-47a5-9586-2bb5e51ce076 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:54.995948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f03c45aa-6af6-47a5-9586-2bb5e51ce076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:55.104622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 707/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 03:59:55.394948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:55.396493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:55.604583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71c2c-8acb-4a53-b70c-347000b14e5a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:55.670257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:55.772693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:55.781210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:55.781210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:55.781210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:55.781210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:55.895535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:55.897195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f6185-ab30-4383-bc43-9cea1e47d122 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:55.916344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 699/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 03:59:57.890764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:57.891411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:57.891411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:57.893016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 708/1407] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 03:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 03:59:57.897952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961953-2440-4287-970d-6d31b81230e8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:57.898377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961953-2440-4287-970d-6d31b81230e8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:57.898484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48961953-2440-4287-970d-6d31b81230e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:57.967790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-48961953-2440-4287-970d-6d31b81230e8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 03:59:57.969018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-48961953-2440-4287-970d-6d31b81230e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:57.971927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:58.069427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 700/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 03:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 03:59:58.359958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 03:59:58.361664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:58.431009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:58.522820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2fdc-1fcf-4016-9be0-e70c3690a74c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 03:59:58.644304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 03:59:58.645375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 03:59:58.645375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 03:59:58.645692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 03:59:58.647055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 03:59:58.667341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 03:59:58.749522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 03:59:58.751480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461affd-1df8-4a62-9d1d-4f3df3b2897f service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 03:59:58.773001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 709/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 03:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:08.067661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=41 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:08.770594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=37 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:15.984003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142efa3-efcd-46b0-b205-0ed3be1e15fe None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:15.984613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142efa3-efcd-46b0-b205-0ed3be1e15fe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:15.984613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142efa3-efcd-46b0-b205-0ed3be1e15fe None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:15.985900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 701/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 31 04:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:15.992277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6470-bb76-4fc6-b315-297627d1f12e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:15.992771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6470-bb76-4fc6-b315-297627d1f12e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:15.992771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e6470-bb76-4fc6-b315-297627d1f12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:16.093037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-db2e6470-bb76-4fc6-b315-297627d1f12e None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:16.094272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-db2e6470-bb76-4fc6-b315-297627d1f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:16.233825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 710/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 31 04:00:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:16.251494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f2d2-02e7-4d52-a6fa-93030e5a7d9a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:16.251773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f2d2-02e7-4d52-a6fa-93030e5a7d9a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:16.252039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f2d2-02e7-4d52-a6fa-93030e5a7d9a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:16.253415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 702/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 31 04:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:16.258247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433377-38ac-49d0-bb8c-dd90a12d7ef9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:16.258533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433377-38ac-49d0-bb8c-dd90a12d7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:16.258804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433377-38ac-49d0-bb8c-dd90a12d7ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:16.334048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:16.337064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c7433377-38ac-49d0-bb8c-dd90a12d7ef9 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:16.338772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c7433377-38ac-49d0-bb8c-dd90a12d7ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:16.473933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 711/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 31 04:00:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:00:18.846734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:18.847317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:18.847317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:18.848527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:00:18.854740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846196b5-5c97-4461-9cbe-fe251d6e99b9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:18.855405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846196b5-5c97-4461-9cbe-fe251d6e99b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:18.855540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846196b5-5c97-4461-9cbe-fe251d6e99b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:18.929482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-846196b5-5c97-4461-9cbe-fe251d6e99b9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:18.930691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-846196b5-5c97-4461-9cbe-fe251d6e99b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:18.987804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:19.033820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:19.330673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:19.333067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:19.509497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6affd0-9d4a-4203-b4e9-b6ae427c2e48 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:19.591147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:19.635106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:19.635711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:19.635982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:19.636244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:19.637842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:19.732538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:19.734134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd476d6-8691-47cf-9541-26814b21d797 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:19.753503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 704/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:20.646921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:20.647924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:20.647924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:20.650150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:20.656237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d13c-bb3a-4ad4-94dc-b46d5c84b82a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:20.656679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d13c-bb3a-4ad4-94dc-b46d5c84b82a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:20.656774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d13c-bb3a-4ad4-94dc-b46d5c84b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:20.748299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7520d13c-bb3a-4ad4-94dc-b46d5c84b82a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:20.749506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7520d13c-bb3a-4ad4-94dc-b46d5c84b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:20.780280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:20.852018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:21.004662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:21.007187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:21.180182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323c6cf-a7c4-4958-acb0-a8a9656878fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:21.225140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:21.311178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:21.311783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:21.312058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:21.312325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:21.313923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:21.407017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:21.408640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a15a0-ee90-47c3-bd34-6a02b21fa63e service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:21.427312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 714/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:22.320709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:22.321231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:22.321231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:22.322574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:22.327930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ee646-7601-4b78-801f-a95347450fcb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:22.328360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ee646-7601-4b78-801f-a95347450fcb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:22.328454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ee646-7601-4b78-801f-a95347450fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:22.383226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:22.405708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0e9ee646-7601-4b78-801f-a95347450fcb None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:22.406837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e9ee646-7601-4b78-801f-a95347450fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:22.517397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:22.673351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:22.675847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:22.846068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c617-2ac0-4bf2-a051-007c5b5a0407 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:22.859286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:22.968977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:22.969585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:22.969858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:22.970147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:22.971863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:23.071442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:23.073076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc10d-7b24-4835-bede-4c5610894ee5 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:23.092252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 707/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:23.956386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:23.957371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:23.957371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:23.959520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:23.967810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af0a8-8664-4c8b-9a8f-7c1e16fd2038 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:23.968534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af0a8-8664-4c8b-9a8f-7c1e16fd2038 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:23.968697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af0a8-8664-4c8b-9a8f-7c1e16fd2038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:23.992708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:24.053441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cc9af0a8-8664-4c8b-9a8f-7c1e16fd2038 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:24.054633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cc9af0a8-8664-4c8b-9a8f-7c1e16fd2038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:24.159079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:24.309729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:24.311408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:24.435417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:24.474098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f87bd9-ee98-4f56-bb20-15ececf6b507 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:24.597640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:24.598146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:24.598422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:24.598680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:24.600356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:24.683503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:24.698334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:24.699972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ed67-b9de-453e-b8ca-62d10b8d89fa service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:24.718484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 717/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:25.585540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:25.586083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:25.586083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:25.587437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:25.593698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c019-8d3f-4cd0-ad2b-916e8e4c7de6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:25.594836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c019-8d3f-4cd0-ad2b-916e8e4c7de6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:25.595175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c019-8d3f-4cd0-ad2b-916e8e4c7de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:25.676544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3936c019-8d3f-4cd0-ad2b-916e8e4c7de6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:25.678385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3936c019-8d3f-4cd0-ad2b-916e8e4c7de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:25.792956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:25.951993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:25.954392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:26.060068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:26.125256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8ebbb-c8ac-4139-8678-08666b1535f2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:26.249895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:26.250418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:26.250691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:26.250956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:26.252612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:26.304120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:26.353235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:26.354947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7258e0-6ac0-4c51-925a-3089ab0c34ad service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:26.374515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 710/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:27.189026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:27.189617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:27.189617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:27.190938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:27.197121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c528-85d1-4e9f-a3ed-0d100ab5eda0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:27.197853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c528-85d1-4e9f-a3ed-0d100ab5eda0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:27.197944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90c528-85d1-4e9f-a3ed-0d100ab5eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:27.284398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9f90c528-85d1-4e9f-a3ed-0d100ab5eda0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:27.285621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9f90c528-85d1-4e9f-a3ed-0d100ab5eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:27.394147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:27.548956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:27.551334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:27.606722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:27.715289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dc22e-4f32-4b04-a775-2cbe6ceecc08 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:27.831633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:27.832249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:27.832525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:27.832792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:27.834466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:27.841914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:27.940997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:27.942554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3c60f-9cc7-428c-b068-4a0913c75bd2 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:27.964707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 720/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:28.759940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:28.760456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:28.760456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:28.761824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:28.767826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052b75b-f17f-4869-a75c-2f377c7e2fcf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:28.768323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052b75b-f17f-4869-a75c-2f377c7e2fcf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:28.768431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052b75b-f17f-4869-a75c-2f377c7e2fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:28.848545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5052b75b-f17f-4869-a75c-2f377c7e2fcf None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:28.849781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5052b75b-f17f-4869-a75c-2f377c7e2fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:28.928293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:28.959047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:29.128389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:29.130613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:29.152297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:29.300299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195293e-aff0-4178-bbf8-d5c141d2e73f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:29.409367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:29.438527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:29.439165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:29.439443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:29.439706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:29.441305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:29.536105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:29.537727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af748327-34e3-4973-9585-3099d9f216b5 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:29.557289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 713/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:30.358728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fff7-072d-4430-9702-98502a503def None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:30.359203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fff7-072d-4430-9702-98502a503def None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:30.359297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1fff7-072d-4430-9702-98502a503def None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:30.360683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:30.366068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b0a03-c546-4416-b526-aa08fd4c25ea None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:30.366587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b0a03-c546-4416-b526-aa08fd4c25ea None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:30.366689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b0a03-c546-4416-b526-aa08fd4c25ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:30.446465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-944b0a03-c546-4416-b526-aa08fd4c25ea None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:30.447726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-944b0a03-c546-4416-b526-aa08fd4c25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:30.494220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:30.557692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:30.712790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1fff7-072d-4430-9702-98502a503def None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:30.714223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-47f1fff7-072d-4430-9702-98502a503def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:30.872991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1fff7-072d-4430-9702-98502a503def None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:30.931314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:30.999494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:31.000141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:31.000422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:31.000693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:31.002358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:31.094872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:31.096498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dcfc5-2bb3-4530-8bf1-263814e6592e service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:31.115353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 723/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:31.943318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:31.943914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:31.943914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:31.945900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:31.954215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64dae-a01c-4973-8a26-eb8700468741 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:31.954643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64dae-a01c-4973-8a26-eb8700468741 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:31.954749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64dae-a01c-4973-8a26-eb8700468741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:32.038746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4cd64dae-a01c-4973-8a26-eb8700468741 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:32.040000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4cd64dae-a01c-4973-8a26-eb8700468741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:32.042040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:32.154151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:32.306290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:32.307860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:32.474706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b382-33ba-4605-85e4-14440ade440d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:32.506134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:32.595716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:32.596234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:32.596508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:32.596774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:32.598379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:32.693106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:32.694737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bc6b-271e-4fc8-a1a6-7ddb8c2739b1 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:32.713167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 716/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:34.540552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:34.541185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:34.541185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:34.542455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:34.549108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21bde-f3ea-4990-a395-dbc478ed8bc2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:34.549813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21bde-f3ea-4990-a395-dbc478ed8bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:34.550010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21bde-f3ea-4990-a395-dbc478ed8bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:34.603735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:34.632256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-78e21bde-f3ea-4990-a395-dbc478ed8bc2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:34.633473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-78e21bde-f3ea-4990-a395-dbc478ed8bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:34.735868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:35.020244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:35.022660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:35.148792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980b04-7c72-47da-9b52-415eb7275d80 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:35.188139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:35.236461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:35.237069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:35.237318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:35.237606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:35.239625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:35.359714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:35.361162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ac062-dca1-4ad9-83d5-6b4680bd10be service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:35.382113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 726/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:37.436354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:37.436914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:37.436914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:37.438270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:37.444870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d584d4-21ac-47fe-969a-063af95274e7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:37.445572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d584d4-21ac-47fe-969a-063af95274e7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:37.445759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d584d4-21ac-47fe-969a-063af95274e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:37.520233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b1d584d4-21ac-47fe-969a-063af95274e7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:37.521504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b1d584d4-21ac-47fe-969a-063af95274e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:37.535889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:37.631933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:37.916818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:37.918407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:38.046821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627afb3-375d-4070-96e5-6fce9e7b9d16 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:38.059451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:38.131054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:38.131660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:38.131930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:38.132193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:38.133832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:38.258698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:38.260189 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce978f1-76bb-47f1-89ea-a8b015c1282f service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:38.284377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 719/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:40.519619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:40.520512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:40.520512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:40.522628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:40.530533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d29d5-aebe-4cbb-b7e2-f7c5001756f5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:40.531089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d29d5-aebe-4cbb-b7e2-f7c5001756f5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:40.531089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d29d5-aebe-4cbb-b7e2-f7c5001756f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:40.581967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:40.608837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-317d29d5-aebe-4cbb-b7e2-f7c5001756f5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:40.610086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-317d29d5-aebe-4cbb-b7e2-f7c5001756f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:40.717281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:40.999811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:41.001351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:41.115430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093633ba-3e8a-4ed6-8ad8-c62dff3651bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:41.154800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:41.196032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:41.196652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:41.196913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:41.197182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:41.198786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:41.466363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:41.467876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93380068-6558-4d73-a0fc-8c7b5bf8f913 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:41.487060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 729/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:43.507492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:43.508078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:43.508078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:43.510123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:43.517884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8eff-3e9a-4841-bde9-96cabd256024 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:43.518311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8eff-3e9a-4841-bde9-96cabd256024 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:43.518442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b8eff-3e9a-4841-bde9-96cabd256024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:43.597523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4b1b8eff-3e9a-4841-bde9-96cabd256024 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:43.598637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4b1b8eff-3e9a-4841-bde9-96cabd256024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:43.621504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:43.707293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:43.992802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:43.995296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:44.112566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2d970-f4aa-4030-a389-b269cc8a52c8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:44.134786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:44.192129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:44.192734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:44.193013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:44.193294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:44.194883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:44.326664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:44.328154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a96350-5fea-4e9c-abfd-378c79fc38d2 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:44.347822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 722/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:46.370641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:46.371633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:46.371633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:46.373130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:46.382049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefff81-3d61-4457-8a16-2c32e500cfd9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:46.382803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefff81-3d61-4457-8a16-2c32e500cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:46.382950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefff81-3d61-4457-8a16-2c32e500cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:46.453481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:46.457613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2eefff81-3d61-4457-8a16-2c32e500cfd9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:46.459682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2eefff81-3d61-4457-8a16-2c32e500cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:46.557435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:46.845532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:46.847962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:46.963906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f6b78-f276-41e2-833c-38d1ef5fa5ce None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:47.018146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:47.043592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:47.044194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:47.044468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:47.044786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:47.046374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:47.164581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:47.166019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30235da4-341a-4875-9766-e70bec0a97af service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:47.184515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 732/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:49.125267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:49.126208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:49.126380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:49.128559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:49.136363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6b7-670a-4ff7-9a78-c5ef252815d1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:49.136846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6b7-670a-4ff7-9a78-c5ef252815d1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:49.137153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6b7-670a-4ff7-9a78-c5ef252815d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:49.218491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5aacd6b7-670a-4ff7-9a78-c5ef252815d1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:49.219752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5aacd6b7-670a-4ff7-9a78-c5ef252815d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:49.250429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:49.327067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:49.618541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:49.621006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:49.737202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4ace-8606-4f4d-99fb-b45e9b21aab3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:49.769719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:49.817946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:49.818542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:49.818816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:49.819180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:49.820788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:49.933764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:49.935306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12ef45-e926-43dc-b08d-47e20dc3621b service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:49.953344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 725/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:51.924101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:51.924934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:51.924934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:51.926297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:51.931911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62898-64eb-42dc-850a-8f287731a739 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:51.932346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62898-64eb-42dc-850a-8f287731a739 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:51.932462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b62898-64eb-42dc-850a-8f287731a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:52.003321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e8b62898-64eb-42dc-850a-8f287731a739 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:52.004530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e8b62898-64eb-42dc-850a-8f287731a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:52.013051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:52.106636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:52.389814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:52.391396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:52.506410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd342a4-e950-4a52-bf84-659b814698bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:52.565934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:52.585781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:52.586388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:52.586668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:52.586940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:52.588590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:52.705624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:52.707071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e8236-bc6b-4c66-aa3a-c220ccb7f077 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:52.725721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 735/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:54.795384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:54.796356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:54.796356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:54.798275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:54.806447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cc725-2cae-44c2-ac2e-d09cf72404b1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:54.806935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cc725-2cae-44c2-ac2e-d09cf72404b1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:54.806935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cc725-2cae-44c2-ac2e-d09cf72404b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:54.878632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-279cc725-2cae-44c2-ac2e-d09cf72404b1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:54.879879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-279cc725-2cae-44c2-ac2e-d09cf72404b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:54.921074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:54.984994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:00:55.277837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:55.280253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:55.399153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269d87b-29e5-49b1-af4f-fe9153e1477e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:55.439230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:55.480415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:55.481018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:55.481297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:55.481567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:55.483266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:55.597085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:55.598461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df44ea-9eb5-4a60-a1ae-454d9d0451f3 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:55.622457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 728/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:00:57.589432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:57.590075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:57.590075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:57.591270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:00:57.597486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5445-559d-4b64-b145-f99eb71343c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:57.598032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5445-559d-4b64-b145-f99eb71343c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:57.598032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5445-559d-4b64-b145-f99eb71343c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:57.669569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-39ad5445-559d-4b64-b145-f99eb71343c6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:00:57.671433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-39ad5445-559d-4b64-b145-f99eb71343c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:57.692162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:57.775392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:00:58.057731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:00:58.060276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:58.168928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d0de0-55d1-4c40-b51f-3097a82380a7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:00:58.236511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:00:58.244962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:00:58.245543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:00:58.245819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:00:58.246089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:00:58.247993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:00:58.359951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:00:58.361390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bae2a5-a140-43cf-8d2d-977a739dabf8 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:00:58.381445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 738/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:00.331009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:00.331495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:00.331612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:00.332938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:00.338460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167475-3d63-4460-97ac-72261f90cdc6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:00.338949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167475-3d63-4460-97ac-72261f90cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:00.339078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9167475-3d63-4460-97ac-72261f90cdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:00.408204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e9167475-3d63-4460-97ac-72261f90cdc6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:00.409410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e9167475-3d63-4460-97ac-72261f90cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:00.464793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:00.516711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:00.798525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:00.800936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:00.932449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382540b-ea76-4b96-affe-3cf8c42b71ca None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:00.987494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:01.015605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:01.016212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:01.016490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:01.016758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:01.018358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:01.135787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:01.137148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb0af8-1900-409d-b35c-bd5cc94e8018 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:01.155291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 731/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:10.514743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=22 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:11.152995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=66 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:20.427824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923cea-eb7f-4eb2-ab4b-ceeeccb1ee11 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:20.428423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923cea-eb7f-4eb2-ab4b-ceeeccb1ee11 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:20.428423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923cea-eb7f-4eb2-ab4b-ceeeccb1ee11 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:20.429710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 740/1471] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:20.436612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232d512-5f4f-4d43-80c5-9ae6d02aeb66 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:20.437398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232d512-5f4f-4d43-80c5-9ae6d02aeb66 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:20.437398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232d512-5f4f-4d43-80c5-9ae6d02aeb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:20.512765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3232d512-5f4f-4d43-80c5-9ae6d02aeb66 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:20.514416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3232d512-5f4f-4d43-80c5-9ae6d02aeb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:20.618679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 732/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:20.928019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:20.928301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:20.928564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:20.929902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 741/1473] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 31 04:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:20.936178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a937-a1ca-4c10-9319-e0a51ac9e513 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:20.936492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a937-a1ca-4c10-9319-e0a51ac9e513 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:20.936878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a937-a1ca-4c10-9319-e0a51ac9e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:21.019809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3dd1a937-a1ca-4c10-9319-e0a51ac9e513 None None] MFA Rules not processed for user `a77972e7e64f47c7a3fe7153841976c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:21.021017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3dd1a937-a1ca-4c10-9319-e0a51ac9e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:21.042783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:21.103648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 733/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 31 04:01:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:21.112674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:21.114186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:21.232557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ee339-1845-49c4-be20-38840df19d6b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:21.313810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:21.314417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:21.314692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:21.314959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:21.316588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:21.373901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:21.433259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:21.434703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ea21c-cfcc-405f-8ddc-f74bf871626f service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:21.453930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 742/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:23.440225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:23.441089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:23.441089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:23.443253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 734/1476] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:23.452062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011182d6-8c7c-4331-bb9f-bca66ce7e601 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:23.452499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011182d6-8c7c-4331-bb9f-bca66ce7e601 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:23.452603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011182d6-8c7c-4331-bb9f-bca66ce7e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:23.532641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-011182d6-8c7c-4331-bb9f-bca66ce7e601 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:23.533861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-011182d6-8c7c-4331-bb9f-bca66ce7e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:23.647100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 743/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:23.917489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:23.920066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:24.061162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38d624-37ca-43a8-8815-0b6529869370 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:24.115412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:24.165760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:24.166592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:24.166942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:24.167344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:24.168401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:24.170784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:01:24.170784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:01:24.170784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:01:24.171971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff878a45-b6c9-4ebe-9471-832df0daaf28 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:24.191846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 735/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 31 04:01:23 2022] GET /identity/v3/projects => generated 2553 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:01:26.102444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:26.103411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:26.103411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:26.105492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 744/1479] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:26.111055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3481e2-072b-4e18-9db3-7a197e29ad51 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:26.111527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3481e2-072b-4e18-9db3-7a197e29ad51 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:26.111652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3481e2-072b-4e18-9db3-7a197e29ad51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:26.185273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8b3481e2-072b-4e18-9db3-7a197e29ad51 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:26.186499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8b3481e2-072b-4e18-9db3-7a197e29ad51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:26.295996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 736/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:26.598116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:26.599931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:26.610423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:26.766877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb255d-0128-4929-b42e-5a130e065889 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:26.862698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:26.900952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:26.901582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:26.901855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:26.902118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:26.903741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:27.007771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:27.009435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eb624-3351-4e8a-b1bc-c76b12bc69bd service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:27.030699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 745/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:29.124374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:29.125268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:29.125268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:29.127469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 737/1482] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:29.133679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187eb03-e2b5-46f9-ae8b-2a0dff1955d0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:29.134436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187eb03-e2b5-46f9-ae8b-2a0dff1955d0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:29.134615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187eb03-e2b5-46f9-ae8b-2a0dff1955d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:29.206153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9187eb03-e2b5-46f9-ae8b-2a0dff1955d0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:29.207379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9187eb03-e2b5-46f9-ae8b-2a0dff1955d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:29.255409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:29.310769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 746/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:29.620912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:29.623303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:29.649484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:29.734424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463a958-3615-4aa0-bc95-5b698719cc56 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:29.810822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:29.811449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:29.811727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:29.811985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:29.813572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:29.889380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:29.927591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:29.929046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281eb85b-53de-4d93-ba72-05918e320f70 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:29.947793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 738/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:31.902877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:31.903447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:31.903447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:31.905146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 747/1485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:31.910778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528d57-f736-4d78-83f1-59fe08fe1179 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:31.911217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528d57-f736-4d78-83f1-59fe08fe1179 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:31.911358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528d57-f736-4d78-83f1-59fe08fe1179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:31.990994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-25528d57-f736-4d78-83f1-59fe08fe1179 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:31.992797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-25528d57-f736-4d78-83f1-59fe08fe1179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:32.095171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 739/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:32.382341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:32.383942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:32.493545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a2b17-7bd9-4089-be74-214a85e973db None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:32.574707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:32.577450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:32.578284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:32.578565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:32.578835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:32.581029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:32.693386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:32.694841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb27c85-13de-414a-88e4-844d60652f2d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:32.713229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 748/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:01:34.767714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:34.768621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:34.768621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:34.770810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 740/1488] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:34.777251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1d5c-9998-4677-906b-d5d21cd8fbd8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:34.777969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1d5c-9998-4677-906b-d5d21cd8fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:34.778172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1d5c-9998-4677-906b-d5d21cd8fbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:34.846847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-06ad1d5c-9998-4677-906b-d5d21cd8fbd8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:34.848097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-06ad1d5c-9998-4677-906b-d5d21cd8fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:34.902727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:34.954245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 749/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:35.252456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:35.254820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:35.296825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:35.373652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbf478-339e-4ccd-a693-740d0f625d65 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:35.468179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:35.468791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:35.469061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:35.469327 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:35.470914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:35.554037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:35.584101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:35.585539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b124803-4c6e-4b0b-bb31-d1ab68a60b08 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:35.603989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 741/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:37.750451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:37.751051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:37.751051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:37.752388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 750/1491] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 31 04:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:01:37.757893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6dc2-179b-4293-8749-98c68a1bb9f6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:37.758353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6dc2-179b-4293-8749-98c68a1bb9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:37.758449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf6dc2-179b-4293-8749-98c68a1bb9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:37.826923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c4cf6dc2-179b-4293-8749-98c68a1bb9f6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:01:37.828208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4cf6dc2-179b-4293-8749-98c68a1bb9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:37.930661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 742/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 31 04:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:01:38.229507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:01:38.231219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:38.360717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e68ab-de3c-4478-8a82-33a7a7406d5f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:01:38.452187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:01:38.452711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:01:38.452979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:01:38.453247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:01:38.454864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:01:38.464410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:38.579214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:01:38.580655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1746f00-9d9e-4094-ac36-e4e26ab22dfd service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:01:38.604149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 751/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:01:47.928542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=82 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:01:48.600303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=50 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:52.646610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:52.647606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:52.647606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:52.739194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:02:52.740424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:52.845713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 743/1494] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:02:52.854213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:52.854775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:52.854775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:52.934139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:02:52.935454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.039737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 752/1495] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:02:53.044627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.045881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.062005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d040fd-b22b-44dd-88ea-aac079aa2424 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:53.131560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:53.168847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:53.169684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:53.169818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:53.170069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:53.170693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:02:53.170988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:53.172032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:53.172032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:02:53.172032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:02:53.172917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:53.184482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 744/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:02:53.193172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.202819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f0efc-112d-4c20-947c-80d4f574aa28 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:53.267450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:53.307157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:53.307784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:53.308061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:53.308448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:53.309098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:53.310874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:53.310874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:02:53.310874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:02:53.312098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:53.369297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 753/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:02:53 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:02:53.374719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.377106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.415376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e5ec0-c2ac-4454-990b-0b9c92e81ae7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:53.521940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:53.522602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:53.522930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:53.523282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:53.524176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:53.525637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:53.525637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:02:53.525637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:02:53.527408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:53.575422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 745/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:02:53 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:02:53.579843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.581389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.647408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c8cf4-b390-4654-9e37-a097fb61fadc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:53.734965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:53.747589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da6895-6270-4788-8686-577584b076fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:53.748203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6895-6270-4788-8686-577584b076fa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:53.748479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6895-6270-4788-8686-577584b076fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:53.748752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da6895-6270-4788-8686-577584b076fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:53.749395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da6895-6270-4788-8686-577584b076fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:53.750480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-94da6895-6270-4788-8686-577584b076fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:53.750480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:02:53.750480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:02:53.751561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da6895-6270-4788-8686-577584b076fa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:53.764276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 754/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:02:53.768523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.770083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.781914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28f814-cd99-40aa-a1f2-4cb33060bba9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:53.808421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:53.887917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:53.888705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:53.888987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:53.889262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:53.889928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:53.890908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:53.890908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:02:53.890908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:02:53.891871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e78d27-e229-46b6-b7f5-7fc686ae5015 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:53.905983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 746/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:02:53.912451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da6895-6270-4788-8686-577584b076fa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:53.913980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94da6895-6270-4788-8686-577584b076fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:53.948114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da6895-6270-4788-8686-577584b076fa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:54.048113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:54.048686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:54.048969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:54.049242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] PATH_INFO: `/v3/projects/8cd16680d6fa49c4a4a1e16a33dc4807/users/04596ed2a5ea4d2db39c3a1e206e243a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:54.072140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd16680d6fa49c4a4a1e16a33dc4807, user_id=04596ed2a5ea4d2db39c3a1e206e243a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:54.073179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:54.073179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:02:54.073179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:02:54.075209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbada4-825a-4777-9d91-3d142b39fd65 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:54.103443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 755/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:02:53 2022] PUT /identity/v3/projects/8cd16680d6fa49c4a4a1e16a33dc4807/users/04596ed2a5ea4d2db39c3a1e206e243a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:02:54.116385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bfc44-2013-4248-b6ac-e9f274cdd371 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:54.116692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bfc44-2013-4248-b6ac-e9f274cdd371 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:54.116968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bfc44-2013-4248-b6ac-e9f274cdd371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:54.301323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-903bfc44-2013-4248-b6ac-e9f274cdd371 None None] MFA Rules not processed for user `04596ed2a5ea4d2db39c3a1e206e243a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:02:54.302476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-903bfc44-2013-4248-b6ac-e9f274cdd371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:54.388876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 747/1502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:02:54.407505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:54.407790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:54.408058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:54.409409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 756/1503] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 31 04:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:02:54.414354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85902-0fc2-43e0-8b05-9a6586632ff1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:54.414640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85902-0fc2-43e0-8b05-9a6586632ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:54.414908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe85902-0fc2-43e0-8b05-9a6586632ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:54.446436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:54.494243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6fe85902-0fc2-43e0-8b05-9a6586632ff1 None None] MFA Rules not processed for user `9a1ecf64ce544ee5aa7839acde57bea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:02:54.495480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6fe85902-0fc2-43e0-8b05-9a6586632ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:54.575817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 748/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 31 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:02:54.589758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:54.591423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:54.646662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:54.709330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2afd49-5f77-4e7f-8714-e171610a01db None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:54.806663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:54.807313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:54.807592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:54.807856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:54.809867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:54.927790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:54.929991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:54.935977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:54.954660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 757/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:02:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:02:55.007046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045403-c20c-476a-a071-6ed6f2f20878 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:55.007519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045403-c20c-476a-a071-6ed6f2f20878 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:55.007638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045403-c20c-476a-a071-6ed6f2f20878 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:55.008895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 749/1506] 10.222.0.18 () {52 vars in 830 bytes} [Wed Aug 31 04:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:02:55.013332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:55.014580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:55.133007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533ae2-5505-4ffc-8810-1721007b0fd6 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:55.244065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:55.244701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:55.244982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:55.245256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] PATH_INFO: `/v3/users/04596ed2a5ea4d2db39c3a1e206e243a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:55.260117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04596ed2a5ea4d2db39c3a1e206e243a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:55.261150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:55.261150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:02:55.261150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:02:55.262404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:55.273920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:55.341183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04596ed2a5ea4d2db39c3a1e206e243a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:55.375395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04596ed2a5ea4d2db39c3a1e206e243a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:55.398112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04596ed2a5ea4d2db39c3a1e206e243a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:55.398893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04596ed2a5ea4d2db39c3a1e206e243a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:55.399347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04596ed2a5ea4d2db39c3a1e206e243a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:55.418335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 758/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:02:55 2022] DELETE /identity/v3/users/04596ed2a5ea4d2db39c3a1e206e243a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:02:55.423184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:55.423464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:55.423727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:55.503862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:02:55.505004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:55.595562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:55.614681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 750/1508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:02:55.626393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:55.627712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:55.747420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e996f-ba82-4382-abdd-36b37f01027f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:55.812507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:02:55.834014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:55.834651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:55.834926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:55.835293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:55.836893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:55.978514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:55.979979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91811d24-aa23-4f1b-b346-6caa2490ac06 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:56.000414 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 759/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:02:56.366729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:02:56.368042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:02:56.379607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f198c-d1aa-4cd0-a9c0-c18ad5894b8c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:02:56.482931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:02:56.483559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:02:56.483828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:02:56.484093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] PATH_INFO: `/v3/projects/8cd16680d6fa49c4a4a1e16a33dc4807` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:02:56.494633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd16680d6fa49c4a4a1e16a33dc4807)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:02:56.495754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:02:56.495754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:02:56.495754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:02:56.496940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:02:56.554856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd16680d6fa49c4a4a1e16a33dc4807', 'request_id': 'req-a537e76a-f8ec-4876-a0ad-297ed389f8e6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:56.567990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd16680d6fa49c4a4a1e16a33dc4807', 'request_id': 'req-a537e76a-f8ec-4876-a0ad-297ed389f8e6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:56.574409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd16680d6fa49c4a4a1e16a33dc4807', 'request_id': 'req-a537e76a-f8ec-4876-a0ad-297ed389f8e6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:56.605175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] The token cache is being invalidate because project 8cd16680d6fa49c4a4a1e16a33dc4807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:02:56.605465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:56.605980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a537e76a-f8ec-4876-a0ad-297ed389f8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:02:56.607344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 751/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:02:56 2022] DELETE /identity/v3/projects/8cd16680d6fa49c4a4a1e16a33dc4807 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:03:05.997741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=62 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:03:06.603052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:04:22.228241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:22.228902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:22.228902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:22.230446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:22.236600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23ba3-e504-4d89-a138-64442ae15e4a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:22.237366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23ba3-e504-4d89-a138-64442ae15e4a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:22.237538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23ba3-e504-4d89-a138-64442ae15e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:22.324375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29c23ba3-e504-4d89-a138-64442ae15e4a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:22.325591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29c23ba3-e504-4d89-a138-64442ae15e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:22.447393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:22.603678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:22.606012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:22.738438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748e889-09bf-4a8a-93c5-2a5eae6b9e32 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:22.744938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:22.745560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:22.745831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:22.746076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:22.754105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:22.755754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee58a5-fc76-4fca-8e5c-bb966154c9b2 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:22.765422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 761/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:23.619536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f84dbb-2f79-4907-bf6b-f01ba3eb0fb9 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:23.620119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f84dbb-2f79-4907-bf6b-f01ba3eb0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:23.620119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f84dbb-2f79-4907-bf6b-f01ba3eb0fb9 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:23.621447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:04:23.626697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50edee-83aa-43ae-9323-65bbd239b900 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:23.627169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50edee-83aa-43ae-9323-65bbd239b900 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:23.627253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50edee-83aa-43ae-9323-65bbd239b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:23.649716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bc50edee-83aa-43ae-9323-65bbd239b900 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:23.650930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bc50edee-83aa-43ae-9323-65bbd239b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:23.660270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:04:23.826562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f84dbb-2f79-4907-bf6b-f01ba3eb0fb9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:23.841706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f84dbb-2f79-4907-bf6b-f01ba3eb0fb9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:23.847171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:23.847766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:23.848043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:23.848311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:23.856810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:23.858400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e929-59fd-466a-bf65-5a4fec79f9d8 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:23.868590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 754/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:24.223785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3630-6bf1-4d0b-9a5c-678c27f391bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:24.224056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3630-6bf1-4d0b-9a5c-678c27f391bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:24.224303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc3630-6bf1-4d0b-9a5c-678c27f391bb None None] PATH_INFO: `/v3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:24.225883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 763/1517] 10.222.0.26 () {54 vars in 925 bytes} [Wed Aug 31 04:04:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:04:24.232573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4b3c-aaa4-4c1a-b163-1c517a683d23 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:24.232855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4b3c-aaa4-4c1a-b163-1c517a683d23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:24.233124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4b3c-aaa4-4c1a-b163-1c517a683d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:24.297934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ae4c4b3c-aaa4-4c1a-b163-1c517a683d23 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:24.299211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ae4c4b3c-aaa4-4c1a-b163-1c517a683d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:24.319681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:04:24.386372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 755/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 31 04:04:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:28.158937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecc862-2c6c-4ea5-85ce-6eb45fec1c12 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:28.159918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecc862-2c6c-4ea5-85ce-6eb45fec1c12 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:28.159918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecc862-2c6c-4ea5-85ce-6eb45fec1c12 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:28.161939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 764/1519] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 31 04:04:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:28.168003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aea39-f81b-4b9b-9ae9-ff07ccdd3db9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:28.168468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aea39-f81b-4b9b-9ae9-ff07ccdd3db9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:28.168573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aea39-f81b-4b9b-9ae9-ff07ccdd3db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:28.193163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bb2aea39-f81b-4b9b-9ae9-ff07ccdd3db9 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:28.194346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bb2aea39-f81b-4b9b-9ae9-ff07ccdd3db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:28.204394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 756/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 31 04:04:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:28.212813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecc862-2c6c-4ea5-85ce-6eb45fec1c12 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:28.229836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecc862-2c6c-4ea5-85ce-6eb45fec1c12 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:28.235674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:28.236270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:28.236547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:28.236820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:28.250229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:28.252649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48c453-9708-4ac3-8f53-fbc4041b3573 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:28.266229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 765/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:35.098746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3519320-3dec-46ad-8a64-02b4d4115dfa None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:35.099785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3519320-3dec-46ad-8a64-02b4d4115dfa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:35.099785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3519320-3dec-46ad-8a64-02b4d4115dfa None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:35.101994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:04:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:35.108174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a8f6-5660-4b23-81a0-025b6545a8e4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:35.108988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a8f6-5660-4b23-81a0-025b6545a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:35.109176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a8f6-5660-4b23-81a0-025b6545a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:35.133984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6b07a8f6-5660-4b23-81a0-025b6545a8e4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:35.135292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b07a8f6-5660-4b23-81a0-025b6545a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:35.146671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:35.305098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3519320-3dec-46ad-8a64-02b4d4115dfa None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:35.320762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3519320-3dec-46ad-8a64-02b4d4115dfa None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:35.326621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:35.327242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:35.327519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:35.327791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:35.336443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:35.338063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403d720-60ec-4987-bdc1-9a175d072e31 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:35.347657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 758/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:04:36.178619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899743-0c27-41f3-9d71-67d7ab6d01f4 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:36.179609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899743-0c27-41f3-9d71-67d7ab6d01f4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:36.180024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6899743-0c27-41f3-9d71-67d7ab6d01f4 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:36.182255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 04:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:36.187316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485f32-67fd-4fd7-9bff-6c50e8514c81 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:36.187789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485f32-67fd-4fd7-9bff-6c50e8514c81 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:36.187892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73485f32-67fd-4fd7-9bff-6c50e8514c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:36.208302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-73485f32-67fd-4fd7-9bff-6c50e8514c81 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:36.209498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-73485f32-67fd-4fd7-9bff-6c50e8514c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:36.221739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 04:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:36.372435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6899743-0c27-41f3-9d71-67d7ab6d01f4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:36.393889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6899743-0c27-41f3-9d71-67d7ab6d01f4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:36.400001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:36.400597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:36.400893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:36.401213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:36.409736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:36.411377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a990cf-b6d3-4c8b-9f8d-30a6019781d8 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:36.421269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 768/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:40.170556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:40.171165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:40.171165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:40.173204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:40.181882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9a8c0-1651-4096-b345-0d06fa839010 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:40.182314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9a8c0-1651-4096-b345-0d06fa839010 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:40.182437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9a8c0-1651-4096-b345-0d06fa839010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:40.208530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-57b9a8c0-1651-4096-b345-0d06fa839010 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:40.209715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-57b9a8c0-1651-4096-b345-0d06fa839010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:40.219274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:40.507780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:40.511098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:40.598882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182af105-2c44-499a-8f0f-e4df94b44df0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:40.602947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:40.603659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:40.603937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:40.604209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:40.613618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:40.615059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02381fa6-deaa-42bc-b436-4fce8c2dfdde service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:40.625453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 761/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:42.567245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:42.568085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:42.568085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:42.570015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:42.575795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976114d-8aa6-457f-bb4c-9ddc6b69d7da None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:42.576250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976114d-8aa6-457f-bb4c-9ddc6b69d7da None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:42.576353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976114d-8aa6-457f-bb4c-9ddc6b69d7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:42.602571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9976114d-8aa6-457f-bb4c-9ddc6b69d7da None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:42.603823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9976114d-8aa6-457f-bb4c-9ddc6b69d7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:42.613035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:42.909855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:42.911997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:42.927625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3ad22-d54d-41fe-8685-f0c84d3e2d60 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:42.931957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:42.932548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:42.932817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:42.933088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:42.942330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:42.943791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f16cc-7644-4a9b-8259-5b2e391cc142 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:42.953680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 771/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:46.579629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:46.580539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:46.580539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:46.582535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:04:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:46.590072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69c373-bc97-4ae7-a236-f756953c00b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:46.590542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69c373-bc97-4ae7-a236-f756953c00b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:46.590656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69c373-bc97-4ae7-a236-f756953c00b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:46.598620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:04:46.613001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0e69c373-bc97-4ae7-a236-f756953c00b5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:46.614210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e69c373-bc97-4ae7-a236-f756953c00b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:46.623214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:04:47.182308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:47.184232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:47.264515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:04:47.280556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895df1-ac19-4af8-9844-c1741a1ccf9a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:47.284432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:47.285553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:47.285553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:47.285553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:47.294818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:47.296294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647d7ac-1301-4f3d-8faa-fb288b83ba0d service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:47.306143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 764/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:49.256713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:49.257285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:49.257285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:49.258711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 04:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:04:49.265140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bf2f4-0863-4a18-a1c4-b27ebab2d399 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:49.265852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bf2f4-0863-4a18-a1c4-b27ebab2d399 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:49.266105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bf2f4-0863-4a18-a1c4-b27ebab2d399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:49.297502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-703bf2f4-0863-4a18-a1c4-b27ebab2d399 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:04:49.298702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-703bf2f4-0863-4a18-a1c4-b27ebab2d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:49.308279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 04:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:04:49.859767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:04:49.861953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:04:49.878803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddb4fe-beac-4393-9ed8-ae3594e6d5a7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:04:49.883504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:04:49.884396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:04:49.884671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:04:49.884938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:04:49.894188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:04:49.895655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889e6fb-0e45-48e8-add4-4ffca55482d9 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:04:49.905459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 774/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:04:59.291591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=11 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:04:59.901378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:12.583382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.584006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.584006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.611776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:12.613009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:12.624829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 766/1540] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:12.633541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc139-fbb3-43fc-8a19-aef8d73baed2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.634125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc139-fbb3-43fc-8a19-aef8d73baed2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.634125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc139-fbb3-43fc-8a19-aef8d73baed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.672135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-398cc139-fbb3-43fc-8a19-aef8d73baed2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:12.673781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-398cc139-fbb3-43fc-8a19-aef8d73baed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:12.689929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 775/1541] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:12.694153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:12.708467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e04ff7-d1a7-45e6-a031-afb45be720f4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:12.717765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:12.718375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.718652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.718917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.719866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:12.720115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:12.721645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:12.733650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30f8e-d501-4190-8b1c-4a32d17c9e92 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.733972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30f8e-d501-4190-8b1c-4a32d17c9e92 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.734316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c30f8e-d501-4190-8b1c-4a32d17c9e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.738017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 767/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:12.746478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:12.757482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12d2f6-422f-40d3-a72b-b7ce87f697c4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:12.764137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d4c30f8e-d501-4190-8b1c-4a32d17c9e92 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:12.765338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d4c30f8e-d501-4190-8b1c-4a32d17c9e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:12.773312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:12.774299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.774800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.775554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.776414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:12.777707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:12.777707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:06:12.777707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:06:12.779399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:12.803330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 776/1543] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:12.817372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.817735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.817930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.860208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:12.861460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:12.862108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 768/1544] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:12.866941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:12.954052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 777/1545] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:12.959208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:12.968979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a485662-ecb1-4d67-8195-136cd928ef0d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:12.985086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:12.985945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:12.986225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:12.986495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:12.987230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:12.987545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:12.991055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:12.991055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:06:12.991055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:06:12.991055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.006200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282401d7-d8a9-4b43-98dd-42f54c848e66 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.013973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 778/1546] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:06:13.020832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.022166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.022477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.022994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.023686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.024585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.027248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.034464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a768c5-09e9-44f5-a3f7-ebee2fb0299e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.055068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.055716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.056031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.056338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.056995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.059175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.068033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 769/1547] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:12 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:13.072737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.088903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab50005-a10c-43eb-9631-eb12d3278ec5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.099115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.100188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.100957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.101470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.103487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.105384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.125411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 770/1548] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:13.129770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.130673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 779/1549] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:13.134708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.145554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70606ac1-9e19-4e7f-aeba-3b2ee5809734 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.271580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.272611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.273111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.273589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.274850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.283052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22af87-595a-42e6-9bc6-7ce5ecfa34fd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.299924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad0d67-5b9f-44e7-9dbb-9b6b73050066 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.313066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 771/1550] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:13.315146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572f58-59ec-4532-81bc-c4f110add766 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.315307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.315624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572f58-59ec-4532-81bc-c4f110add766 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.315994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12572f58-59ec-4532-81bc-c4f110add766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.316432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.316880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.317307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.318156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.319369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:13.319369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:06:13.319369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:06:13.320956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53171231-9a0d-4863-88ff-0978810246e2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.338934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-12572f58-59ec-4532-81bc-c4f110add766 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.340359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-12572f58-59ec-4532-81bc-c4f110add766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.349928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 772/1551] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.351307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7048-bdf7-46c5-975b-24fabdc7e827 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.351515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 780/1552] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:13.351648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7048-bdf7-46c5-975b-24fabdc7e827 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.351991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7048-bdf7-46c5-975b-24fabdc7e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.353515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.354039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.354507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.381677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7ccb7048-bdf7-46c5-975b-24fabdc7e827 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.382904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7ccb7048-bdf7-46c5-975b-24fabdc7e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.383073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.384460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.391239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 773/1553] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.392717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.392877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 781/1554] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.393059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.393436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.393818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.403592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4e16f-e110-40a8-bbfb-f8ffee3432d3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.408312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.408966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.409241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.409508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] PATH_INFO: `/v3/projects/985d43160bda4a45a77225f3387adb2f/users/722e6b0dd66541bcbeaeeebd5a02bec4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.416436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.417624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.423921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=985d43160bda4a45a77225f3387adb2f, user_id=722e6b0dd66541bcbeaeeebd5a02bec4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.425532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 774/1555] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.426309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.426741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852ee75-1d0b-4a5e-962f-6ee7931e1d4c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.432550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:13.437690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8920c-572d-48ed-a470-79a370464f23 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.442333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 782/1556] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:13 2022] PUT /identity/v3/projects/985d43160bda4a45a77225f3387adb2f/users/722e6b0dd66541bcbeaeeebd5a02bec4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:13.444964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff24ca-9eef-46d4-bc1b-2a69db6d8847 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.445254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff24ca-9eef-46d4-bc1b-2a69db6d8847 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.445524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff24ca-9eef-46d4-bc1b-2a69db6d8847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.465348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4aff24ca-9eef-46d4-bc1b-2a69db6d8847 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.466571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4aff24ca-9eef-46d4-bc1b-2a69db6d8847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.472172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:13.497569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.498187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.498467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.498735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.499407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.501199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f3c42-8af6-482d-9cbd-5b40ec00b119 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.511015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 775/1557] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:13.512228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068f214-23ab-4e10-b16e-a178ddb1adf7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.512510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068f214-23ab-4e10-b16e-a178ddb1adf7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.512781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068f214-23ab-4e10-b16e-a178ddb1adf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.527986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 783/1558] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.529337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d783f-d337-4e77-bd5d-bf5412098f89 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.529632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d783f-d337-4e77-bd5d-bf5412098f89 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.529901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d783f-d337-4e77-bd5d-bf5412098f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.532808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0068f214-23ab-4e10-b16e-a178ddb1adf7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.534036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0068f214-23ab-4e10-b16e-a178ddb1adf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.541947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 776/1559] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.543159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e776-e788-4c87-8fca-97fad2181a36 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.543440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e776-e788-4c87-8fca-97fad2181a36 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.543701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457e776-e788-4c87-8fca-97fad2181a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.554744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-916d783f-d337-4e77-bd5d-bf5412098f89 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.555980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-916d783f-d337-4e77-bd5d-bf5412098f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.562611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a457e776-e788-4c87-8fca-97fad2181a36 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.563884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a457e776-e788-4c87-8fca-97fad2181a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.571015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 784/1560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.571015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113551a-4b07-46c0-9eef-e843d1fae867 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.571015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113551a-4b07-46c0-9eef-e843d1fae867 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.571015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113551a-4b07-46c0-9eef-e843d1fae867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.572417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 777/1561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.573495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.573828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.574028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.586945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b113551a-4b07-46c0-9eef-e843d1fae867 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.588666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b113551a-4b07-46c0-9eef-e843d1fae867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.593135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.594286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.596888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 785/1562] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.598076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8572b-f0e9-47b5-b204-c9e09d693c11 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.598353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8572b-f0e9-47b5-b204-c9e09d693c11 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.598645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8572b-f0e9-47b5-b204-c9e09d693c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.603148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 778/1563] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.604006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.619021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bd5be-007f-4c44-ab37-02d749ec75c7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.625144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.625766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.626113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.626320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.626956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.627332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b3f8572b-f0e9-47b5-b204-c9e09d693c11 None None] MFA Rules not processed for user `722e6b0dd66541bcbeaeeebd5a02bec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.628228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3f8572b-f0e9-47b5-b204-c9e09d693c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.628638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.638716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 779/1564] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:13.639454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.649086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94283f47-c7ab-45eb-8fc1-b75b8b6652ac admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.653736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.654298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.654575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.654840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.655502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.655798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.657402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e86c7-2d55-4e56-b844-5af670a44e86 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.669595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 780/1565] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.670746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.671007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.671339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.690864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.692111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.699538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 786/1566] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.700049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 781/1567] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.700622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.700910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.701018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.701182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.713752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c0c51-451f-419e-a79c-4c2daa3098b0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.718525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.719199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.719506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.719786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.720464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.720798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.722445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.726965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:13.728257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:13.744226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 782/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.744226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.744507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 787/1569] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:13.744507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.753008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb015e-3bf7-4a3e-a1e4-94a1bbc83d7f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.753977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea279483-5707-4641-b795-23eea2c22fad admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.757751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.758345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.758561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.758677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.758929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.759201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.759478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.759595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.759748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.759861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.760383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.760667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.761471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.762251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.773566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 788/1570] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.774249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.781148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 783/1571] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.783877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.787228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16bc8-af57-4e06-899d-d7fdfd9fa096 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.792804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.793471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.793588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a66de-08c3-41ec-ac2e-407455528a3e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.793759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.794032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] PATH_INFO: `/v3/projects/dfea8632ec6f4cb5ac84e58d4fe10e6a/users/7103b8b46630401889a8e040c0e9ad62/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.798039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.798622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.798896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.799198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.799824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.801729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d675-35b8-4200-ac34-dff68f3ae16c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.809363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfea8632ec6f4cb5ac84e58d4fe10e6a, user_id=7103b8b46630401889a8e040c0e9ad62, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.812050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.826900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 789/1572] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:13 2022] PUT /identity/v3/projects/dfea8632ec6f4cb5ac84e58d4fe10e6a/users/7103b8b46630401889a8e040c0e9ad62/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:13.833814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 784/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:13.833979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.834081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b02b-4943-4a49-b091-3f4fd912a8da None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.834081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b02b-4943-4a49-b091-3f4fd912a8da None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.834081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b02b-4943-4a49-b091-3f4fd912a8da None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.834081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 785/1574] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 31 04:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:06:13.835126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b02b-4943-4a49-b091-3f4fd912a8da None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.879067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:13.896655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e172f3-81c9-4fa4-9d77-aa0019d213f0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.900245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b02b-4943-4a49-b091-3f4fd912a8da None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.901199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.901806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.902085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.902372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.903001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.903332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.904672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.904929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.905266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.905540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.905803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.906430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.908487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.914016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:13.917377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 790/1575] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.918278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.931332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d454-e2aa-43b3-907b-1c4733521000 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:13.936494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:13.937098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:13.937365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:13.937628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:13.938288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:13.938605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:13.939933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 786/1576] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:13.940204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:13.940583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.954000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 791/1577] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:13.961698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:13.970739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d811df-fe37-4402-92f9-2a1f5f5cdfa1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.005530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2ec2d-7ab7-4fe6-82d6-9262536f7dc9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.010413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.011047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.011328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.011596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.012233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.014151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5992c-b0d6-4e2c-8246-6101f2b51615 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.031417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.032122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.032412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.032649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.033285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.035295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.044862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 787/1578] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.044862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0205b-d683-4165-928d-1c033c7955ec None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.044862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0205b-d683-4165-928d-1c033c7955ec None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.044862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0205b-d683-4165-928d-1c033c7955ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.063005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a2c0205b-d683-4165-928d-1c033c7955ec None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:14.064449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a2c0205b-d683-4165-928d-1c033c7955ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:14.065106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 792/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:13 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.065719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.130183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 788/1580] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:14.131819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.132105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.132377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.135535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e498-aa5e-436c-8dcc-42c9d267f165 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.140162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.140771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.141048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.141317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.141990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.143989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.166614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 793/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.168065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.170519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:14.171780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:14.178031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c484-d16e-4f87-8dea-760d0543ec50 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.182842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.183433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.183708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.184029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.184640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.186540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.215229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 794/1582] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.215951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.223654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:14.234869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:14.241541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 789/1583] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:14.242343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.252275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b65c9e-8129-4005-96ed-1e7b85f9afcd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.292117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fedc4d-9484-4302-904d-31562fbe365d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.297947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.298583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.298857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.299180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.299816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.301767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.311732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.312384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.312639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.313267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.313799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.315891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.325335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 795/1584] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.326128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.335552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c982f5a-d901-4d6e-acb4-be945e5b0dbb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.340683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.341283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.341559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.341827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.342454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.344384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.350413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 790/1585] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.350413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.368190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 796/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.369006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.411724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da260-8931-4258-a241-40023e9ebbd9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.416876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.417422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.417698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.417968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.418602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.420577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.429439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f587-c13a-4a60-a291-d73d94d1668c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.446605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 791/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.446605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.453595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35620-91a4-404a-9c25-0ba118e35869 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.458682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.459281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.459555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.460232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.460871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.463012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.473764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 792/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.474548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.484021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f3da-2b80-49f4-b1a4-0bcd5f1d1512 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.484654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.485325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.485598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.485867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.488658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.489241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.489521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.490117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.490743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.492693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.495294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.496726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.506962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 797/1589] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:14.507975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.522702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 793/1590] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.522702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.536069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc250ae-d8b9-4ca5-8ccf-6a9239ef3930 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.541501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.542253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.542635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.542999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.543760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.545729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.557100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 794/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.558013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.562999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:14.569985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15753a-dbc9-4d89-b5e0-0ab4b8fd5981 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.575409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.576354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.576534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.576925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.577572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.579632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.591816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6d8ca-86cb-4e08-8bee-c6e49855ed12 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.598026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.598779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.599194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.599567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.603805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 795/1592] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:14 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:14.604702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.615513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5e280-d623-4d70-b234-a8c758da6c06 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.620530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.621187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.621974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.621974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.622413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.624244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.626774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:14.636080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 796/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.637012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.649002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393fb0d-381a-4145-9cac-bb48ab5aeeb6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.654358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.655070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.655420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.655752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.656459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.658294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.673003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 797/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.673003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.678724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.680757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.683783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b5f90-bb7f-4bc0-8f7c-89ca293568af admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.689892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.690560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.690953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.691295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.692011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.693569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 798/1595] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:14.693789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.694370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.710317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e88e8-fb50-4750-9191-f771b1094bd2 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.711324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 798/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.711324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.719553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa358d34-eb6d-43a4-9df3-d2bd5fa610a5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.725058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.725375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.725880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.726207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.726866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.727516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.728671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.729211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.729542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.729958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.730838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.733301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.744370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 799/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.744370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.754979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 799/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.755216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764c5b5-f8f9-45da-b367-1b11f29bb7d4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.756872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.760424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.762086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.762441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.762893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.763633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.765547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.768812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b42-51bc-4e03-af4d-de8fa6b2ca0a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.774801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.776017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.776302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.776676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.777324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.779802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.780338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 800/1599] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.781409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.794034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892e1ca-a9ef-4165-86e0-dd4e3ecc51af admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.795596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 800/1600] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.800048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.800503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.801113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.801461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.801822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.802642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.804717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.814550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac476a5-171d-4208-9617-a73a349b14da admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.820035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.820626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.820951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.821227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] PATH_INFO: `/v3/projects/2af1f38c50c24416910c2c459fba18ed/users/1a061fce2cbc40aeacf1d442f65f86c5/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.823003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 801/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:14.823522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.836029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3cc8b-1fca-4304-81ea-ed53327ffd4a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:14.842725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:14.843488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:14.843829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:14.843960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af1f38c50c24416910c2c459fba18ed, user_id=1a061fce2cbc40aeacf1d442f65f86c5, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.844224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] PATH_INFO: `/v3/projects/53dee0538d6744a3983ccd114dcc6de4/users/85aba434ffdc44a191f55c4881f13327/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:14.847531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.865763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53dee0538d6744a3983ccd114dcc6de4, user_id=85aba434ffdc44a191f55c4881f13327, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:14.867248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:14.867248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:06:14.867248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:06:14.869377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:14.870241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 801/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:14 2022] PUT /identity/v3/projects/2af1f38c50c24416910c2c459fba18ed/users/1a061fce2cbc40aeacf1d442f65f86c5/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:14.871392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.911831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 802/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:14 2022] PUT /identity/v3/projects/53dee0538d6744a3983ccd114dcc6de4/users/85aba434ffdc44a191f55c4881f13327/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:14.915130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:14.987573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e33e8-45f3-403f-9910-3a727b9cc8e6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.011043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a05f44-35af-4563-8096-58cd36485e99 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.017770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.018490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.018834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.019192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.019900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.021569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.033483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 803/1604] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:15.038032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.048874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:15.049765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da423-9311-4215-8186-77ce2e723d67 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.055462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.056392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.057158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.057900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] PATH_INFO: `/v3/projects/fdbbb020a1944de2a8fa16379f8cec7d/users/f80c8c1d6af7450eb8f936df231ad63b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.063885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.064909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.065261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.065725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.066637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.068861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.075009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbbb020a1944de2a8fa16379f8cec7d, user_id=f80c8c1d6af7450eb8f936df231ad63b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.077829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a0576-21e1-449f-8cb8-344dbccf1983 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.083507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 802/1605] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:15.083507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.092393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f781b2-bd3e-4780-ae39-4a6a77d2f249 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.097161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 804/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:15 2022] PUT /identity/v3/projects/fdbbb020a1944de2a8fa16379f8cec7d/users/f80c8c1d6af7450eb8f936df231ad63b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:15.100794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923eacd-c9ef-4554-af34-a34f8ae25b27 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.100794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923eacd-c9ef-4554-af34-a34f8ae25b27 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.101029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923eacd-c9ef-4554-af34-a34f8ae25b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.132128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6923eacd-c9ef-4554-af34-a34f8ae25b27 None None] MFA Rules not processed for user `1a061fce2cbc40aeacf1d442f65f86c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.133363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6923eacd-c9ef-4554-af34-a34f8ae25b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.168045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.169069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.169369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.169624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] PATH_INFO: `/v3/projects/66bcbde4eda6417ea15dca2371c28540/users/956b08da3ba944d382877cf266389127/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.178401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:15.190616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bcbde4eda6417ea15dca2371c28540, user_id=956b08da3ba944d382877cf266389127, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.193266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683193f-86f3-40c6-86de-125b5359158b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.209184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 805/1607] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.214441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 803/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:15 2022] PUT /identity/v3/projects/66bcbde4eda6417ea15dca2371c28540/users/956b08da3ba944d382877cf266389127/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:15.214441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742f38-4b89-48ce-ac74-466f83c60923 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.214441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742f38-4b89-48ce-ac74-466f83c60923 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.214441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b742f38-4b89-48ce-ac74-466f83c60923 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.215868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91299a-4d9d-4650-b435-f1ca8eb03468 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.216406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91299a-4d9d-4650-b435-f1ca8eb03468 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.216888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91299a-4d9d-4650-b435-f1ca8eb03468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.218423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 804/1609] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 31 04:06:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:06:15.219774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b742f38-4b89-48ce-ac74-466f83c60923 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.268728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4c91299a-4d9d-4650-b435-f1ca8eb03468 None None] MFA Rules not processed for user `85aba434ffdc44a191f55c4881f13327`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.270297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4c91299a-4d9d-4650-b435-f1ca8eb03468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.311401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b742f38-4b89-48ce-ac74-466f83c60923 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.316021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.316681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.316944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.317224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] PATH_INFO: `/v3/projects/4db5abd0cdf949239e92bc4559a64a34/users/d7224a0ed6334b36be867f5b227fd6ce/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.333047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db5abd0cdf949239e92bc4559a64a34, user_id=d7224a0ed6334b36be867f5b227fd6ce, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.335810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.351115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 805/1610] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:15 2022] PUT /identity/v3/projects/4db5abd0cdf949239e92bc4559a64a34/users/d7224a0ed6334b36be867f5b227fd6ce/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:15.352010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.374471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 806/1611] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.376020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa7b-6e40-4526-b055-b4163724737a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.376306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa7b-6e40-4526-b055-b4163724737a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.376573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa7b-6e40-4526-b055-b4163724737a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.403411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-357eaa7b-6e40-4526-b055-b4163724737a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.404726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-357eaa7b-6e40-4526-b055-b4163724737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.425220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:15.449334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120ad8-19ec-4fa8-b0b0-b96282a3b977 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.455888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.457681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.458558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.460015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] PATH_INFO: `/v3/projects/490f695240294bbaa99f5eaa1e32b977/users/77fb76da34bf44a1b214a08bb4b8e51b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.481545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=490f695240294bbaa99f5eaa1e32b977, user_id=77fb76da34bf44a1b214a08bb4b8e51b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.484773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca14cc-eae3-48cd-b725-ad98333ff3dc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.489360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 807/1612] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.492699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.493043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.493786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.509484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 806/1613] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:15 2022] PUT /identity/v3/projects/490f695240294bbaa99f5eaa1e32b977/users/77fb76da34bf44a1b214a08bb4b8e51b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:15.511216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3048-ec8f-45bb-9895-193ab75de260 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.511625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3048-ec8f-45bb-9895-193ab75de260 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.511892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae3048-ec8f-45bb-9895-193ab75de260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.527531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.529046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.538107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-00ae3048-ec8f-45bb-9895-193ab75de260 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.540688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00ae3048-ec8f-45bb-9895-193ab75de260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.622148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 807/1614] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.624471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f76694-00eb-4cb8-98f8-41fb23c86480 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.625185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f76694-00eb-4cb8-98f8-41fb23c86480 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.625462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f76694-00eb-4cb8-98f8-41fb23c86480 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.627330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 808/1615] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 31 04:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:06:15.628458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c836e-c26f-4966-8ef6-6bb49ed7f6b1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.628859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c836e-c26f-4966-8ef6-6bb49ed7f6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.629123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c836e-c26f-4966-8ef6-6bb49ed7f6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.636124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 808/1616] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.637473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.660399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:15.673770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6e4c836e-c26f-4966-8ef6-6bb49ed7f6b1 None None] MFA Rules not processed for user `d7224a0ed6334b36be867f5b227fd6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.675842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6e4c836e-c26f-4966-8ef6-6bb49ed7f6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.719687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33b9b-0ab2-420a-abfe-635f91b143a2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.724996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.725881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.726162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.726484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.744525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.746380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a43d0-0a50-4f29-8109-ac7d4c30bf12 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.759864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 809/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:15.761397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.761680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.761950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.764708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 809/1618] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.766634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.767660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.768095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.784065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.785344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.794147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 810/1619] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.795233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.807451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:15.813393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:15.835457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 810/1620] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:15.845100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.860830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdab835-5052-457d-a07c-53e93fb2b949 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.864643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.865578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.865915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.866174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.868965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d00ff-7fe9-45df-99d5-63b88bfb414a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.881105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.881903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.882721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.883293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.883778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.884289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.908020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.909864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.910010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 811/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:15.910990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.928927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6be3-701b-426f-ac1c-eef79a53ae8a service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.933598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 811/1622] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:15.933598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.939005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.940806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.941167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.941512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.942358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.944510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:15.955568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa11b-a30e-4872-afce-35c519c74303 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:15.970577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:15.971227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:15.971829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:15.972224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:15.983566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 812/1623] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:15 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:15.984679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:15.992898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:15.994736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.012935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 812/1624] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:16.013787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.024476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e47c-3a35-49cb-a024-0a4438443dd8 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.058934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:16.124192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6ecdf-7c60-45e2-b289-80cfc321b9bd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.133102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.135393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.135866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.136144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.141811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.142825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.143276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.143647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.144390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.149741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.169529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:16.198521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 813/1625] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:16 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:16.200511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.214795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.219727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.220689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182a5ff-7aa2-40b2-8e6e-c561ff46cd3c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.225987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.226597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.226893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.227163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.228036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.229987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.247893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 813/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:16.276748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 814/1627] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:16.281116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.296222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9969cb-d534-4d72-b680-3cd325f3f9d6 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.307976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.308697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.310115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.311268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] PATH_INFO: `/v3/projects/0550987a1b1344e4b780627dc2c16214/users/1c8df4e6ae794fa5b5797891d49e58ae/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.369794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0550987a1b1344e4b780627dc2c16214, user_id=1c8df4e6ae794fa5b5797891d49e58ae, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.377756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.416983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 814/1628] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:16 2022] PUT /identity/v3/projects/0550987a1b1344e4b780627dc2c16214/users/1c8df4e6ae794fa5b5797891d49e58ae/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:16.420951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.435737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5c345-d8e7-4546-8d93-6391eaab6a7e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.555194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.556137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.556569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.557007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.558201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.561111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f13773-f23a-43b4-889b-54de0c5c030f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.577315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 815/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:16.582314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.601931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a8c07-f2d0-445d-a3e3-88981cb6fd6f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.614432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.615477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.615757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.616047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] PATH_INFO: `/v3/projects/0550987a1b1344e4b780627dc2c16214/users/1c8df4e6ae794fa5b5797891d49e58ae/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.618696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0550987a1b1344e4b780627dc2c16214, user_id=1c8df4e6ae794fa5b5797891d49e58ae, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:16.621400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:16.650156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 815/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:16 2022] PUT /identity/v3/projects/0550987a1b1344e4b780627dc2c16214/users/1c8df4e6ae794fa5b5797891d49e58ae/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:16.737479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.737479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.737479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.737479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] MFA Rules not processed for user `1c8df4e6ae794fa5b5797891d49e58ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:16.749705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:16.758206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:16.876210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361468f-5200-40b6-a00f-daf2f5750d62 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:16.885454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:16.886233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:16.887252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:16.887570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:16.955492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 816/1631] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:16.968357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:17.003573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:17.035063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:17.036704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dfaa3-7cb3-467c-b96f-fbf9efbaa696 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:17.065585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 816/1632] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:17.126238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919eaec-112e-40b5-9dfe-4ab953a67957 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:17.136722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:17.137508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:17.137911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:17.138334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:17.153785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:17.155493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:17.173717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16713f-cd1a-410d-b7df-ce211e762103 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:17.174011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16713f-cd1a-410d-b7df-ce211e762103 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:17.174302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16713f-cd1a-410d-b7df-ce211e762103 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:17.176046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 817/1633] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 31 04:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:06:17.181588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:17.181868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:17.182139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:17.186896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 817/1634] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:17.222057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:17.225276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:17.267195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 818/1635] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:06:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:17.285587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:17.309140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9b33b-5042-4968-ad67-1a4ea288c6a9 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:17.313818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:17.315608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:17.316028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:17.316635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:17.333055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:17.335646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5ffac-3f30-41a0-89b9-99e0a8795d34 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:17.364551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 818/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:17.714247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:17.751498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:17.828608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2b884-dc79-4dc1-9d4b-cb78bac59e23 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:17.834287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:17.835702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:17.836210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:17.836685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] PATH_INFO: `/v3/users/1a061fce2cbc40aeacf1d442f65f86c5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:17.838094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a061fce2cbc40aeacf1d442f65f86c5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:17.839613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:17.839613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Aug 31 04:06:17.839613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: Aug 31 04:06:17.840921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:18.003303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a061fce2cbc40aeacf1d442f65f86c5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.006187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a061fce2cbc40aeacf1d442f65f86c5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.051225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a061fce2cbc40aeacf1d442f65f86c5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.080191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a061fce2cbc40aeacf1d442f65f86c5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.085904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a061fce2cbc40aeacf1d442f65f86c5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.153844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 819/1637] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:17 2022] DELETE /identity/v3/users/1a061fce2cbc40aeacf1d442f65f86c5 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:18.158809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:18.159384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:18.159698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:18.219528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:18.221980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:18.385076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 819/1638] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:18.416645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:18.420176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:18.427804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:18.433748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:18.456055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb8b1e-f9f6-47ae-9602-85cbff4e8282 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:18.461687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:18.462412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:18.462769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:18.463183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] PATH_INFO: `/v3/users/d7224a0ed6334b36be867f5b227fd6ce` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:18.464688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7224a0ed6334b36be867f5b227fd6ce)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:18.467740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:18.610157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7224a0ed6334b36be867f5b227fd6ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.643962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9f3d6-a6df-4450-968e-3a2d650f0f54 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:18.659409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:18.660800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7224a0ed6334b36be867f5b227fd6ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.663086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:18.663382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:18.664220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:18.670384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7224a0ed6334b36be867f5b227fd6ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.673342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7224a0ed6334b36be867f5b227fd6ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.674229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f93ed8a8-4b27-4d99-ae8f-321994e6b82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7224a0ed6334b36be867f5b227fd6ce'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:18.696800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:18.701458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 820/1639] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:18 2022] DELETE /identity/v3/users/d7224a0ed6334b36be867f5b227fd6ce => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:18.707007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:18.707391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:18.707633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:18.707793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:18.727542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 820/1640] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:18.761679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:18.763067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:18.817168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:18.903668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 821/1641] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:18.927572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:18.930092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:18.952297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c4d4-f354-4e95-8522-54bc261cb825 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:19.070710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:19.099698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:19.100677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:19.101933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:19.102809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:19.121031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:19.123063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:19.137408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 821/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:19.916693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:19.919125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:19.939849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a6f57-834b-4313-a58d-87e971e59ff6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:19.954178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:19.955461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:19.956403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:19.956852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] PATH_INFO: `/v3/users/722e6b0dd66541bcbeaeeebd5a02bec4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:19.958714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=722e6b0dd66541bcbeaeeebd5a02bec4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:19.962809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:20.075138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722e6b0dd66541bcbeaeeebd5a02bec4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.124023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722e6b0dd66541bcbeaeeebd5a02bec4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.149359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722e6b0dd66541bcbeaeeebd5a02bec4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.150132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722e6b0dd66541bcbeaeeebd5a02bec4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.150765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-58de5ce8-e107-4285-9484-2b8a451f5907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722e6b0dd66541bcbeaeeebd5a02bec4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.188151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:20.193677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.195543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 822/1643] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:19 2022] DELETE /identity/v3/users/722e6b0dd66541bcbeaeeebd5a02bec4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:20.199712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.200003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.200273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.228912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486e5a-2b7f-41aa-b921-e5c6ac244f6e service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:20.243346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:20.246423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.374692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 823/1644] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:20.375723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:20.378063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.402592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d036e-8d6d-4726-87cb-aba3e08612e2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:20.419754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:20.420112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:20.420488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.420771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.420865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.421044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] PATH_INFO: `/v3/projects/2af1f38c50c24416910c2c459fba18ed` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.421129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.421307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] PATH_INFO: `/v3/projects/4db5abd0cdf949239e92bc4559a64a34` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.422608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db5abd0cdf949239e92bc4559a64a34)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:20.422706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af1f38c50c24416910c2c459fba18ed)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:20.423748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 04:06:20.423748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Aug 31 04:06:20.423748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: Aug 31 04:06:20.424790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:20.424902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:20.499932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:20.544928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db5abd0cdf949239e92bc4559a64a34', 'request_id': 'req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.548795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af1f38c50c24416910c2c459fba18ed', 'request_id': 'req-4bdb8e00-46fa-45d5-9de1-0ee569589db4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.559509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af1f38c50c24416910c2c459fba18ed', 'request_id': 'req-4bdb8e00-46fa-45d5-9de1-0ee569589db4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.567589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af1f38c50c24416910c2c459fba18ed', 'request_id': 'req-4bdb8e00-46fa-45d5-9de1-0ee569589db4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.583972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db5abd0cdf949239e92bc4559a64a34', 'request_id': 'req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.593112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db5abd0cdf949239e92bc4559a64a34', 'request_id': 'req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.608657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] The token cache is being invalidate because project 2af1f38c50c24416910c2c459fba18ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:20.609405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.611388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.614115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] The token cache is being invalidate because project 4db5abd0cdf949239e92bc4559a64a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:20.614283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 822/1645] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:20 2022] DELETE /identity/v3/projects/2af1f38c50c24416910c2c459fba18ed => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:20.614380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.615840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:20.618846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.622788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-19daa07a-b02f-42d9-bdea-b2145b0f6b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.625254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 824/1646] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:20 2022] DELETE /identity/v3/projects/4db5abd0cdf949239e92bc4559a64a34 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:20.626433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.626719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.626991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.667314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:20.668633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.717916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb8e00-46fa-45d5-9de1-0ee569589db4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:20.723339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:20.724151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.724434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.724705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.726831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.770016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:20.802920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 825/1647] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:20.804572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:20.810723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.839606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b90a61-4d71-4605-a7cd-766b15ccfc64 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:20.852426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:20.853093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.853374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.854185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:20.855705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b9a92-e62c-4178-994c-51d9739cca60 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:20.855825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] PATH_INFO: `/v3/users/1c8df4e6ae794fa5b5797891d49e58ae` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.858204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8df4e6ae794fa5b5797891d49e58ae)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:20.868333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:20.877724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 823/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:20.885096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:20.885381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:20.887726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:20.943441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:20.946468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:20.965889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 824/1649] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:20.970054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:20.981219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8df4e6ae794fa5b5797891d49e58ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:20.996543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733e38-3020-4c26-b75b-4028f1257f47 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.017579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.022671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.023992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.024493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.027579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:21.028038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.029909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.054922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8df4e6ae794fa5b5797891d49e58ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:21.064907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 825/1650] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:21.069315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8df4e6ae794fa5b5797891d49e58ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:21.069621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.073233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8df4e6ae794fa5b5797891d49e58ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:21.074265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8df4e6ae794fa5b5797891d49e58ae'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:21.090954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc048ec-d996-42d6-9df9-fc09d45cd1e1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.097518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 826/1651] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:20 2022] DELETE /identity/v3/users/1c8df4e6ae794fa5b5797891d49e58ae => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:21.117439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.118289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.118510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.118738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.119466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.121981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.175518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 826/1652] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:21 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:21.180289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.182880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:21.271263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:21.350904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b988045-9ce2-4b71-8746-e68653219617 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.444444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.445232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.445626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.446055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.446774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.449394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.483731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 827/1653] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:21 2022] POST /identity/v3/users => generated 303 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:21.488148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.506955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc549d-ac01-4cbf-80fb-201f81f106d5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.512871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.513437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.513723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.513997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.515202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.516913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.532905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 827/1654] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:21.534702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.551566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d9288-84e9-4ea1-9ead-f82070ac2e7d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.558660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.561028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.561484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.561930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.563174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.566892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649ac3d-c7ec-47de-834b-f03ad61c1377 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.582610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 828/1655] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:21.582955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.603444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ae79-5bc8-4155-806b-d9fbf734506f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:21.612516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:21.613657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.614149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.614578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] PATH_INFO: `/v3/projects/31535257f93f4e32bff3b4f045fa55f4/users/19224c3ba9564f44be82ef2d8653b161/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.641593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31535257f93f4e32bff3b4f045fa55f4, user_id=19224c3ba9564f44be82ef2d8653b161, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:21.645299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:21.688871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:21.689419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:21.689821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:21.697715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 828/1656] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:21 2022] PUT /identity/v3/projects/31535257f93f4e32bff3b4f045fa55f4/users/19224c3ba9564f44be82ef2d8653b161/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:21.752769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] MFA Rules not processed for user `19224c3ba9564f44be82ef2d8653b161`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:21.754345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:21.888575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:21.907543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 829/1657] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:21.995660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec44d-e408-4690-89d4-23e19192172c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.004533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.005172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.005500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.005821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.006525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.008591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.055359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.063547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 829/1658] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:21 2022] POST /identity/v3/projects => generated 403 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:22.069316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8313a-16b7-4e35-8d1e-db698e38e8e1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.144820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:22.163815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.165278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.165626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.166388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.167629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.171315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.226522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 830/1659] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:22 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:22.230549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.244897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f4f9-20d4-4e30-8820-b4be586ddc59 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.263065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.263781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.264178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.264458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.266982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.269546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.285083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.291110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:22.301661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 830/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:22.302571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.322538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262c1e0-7ee8-412a-93f7-021ca99f5340 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.344632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d2119c-da08-40b8-be27-cc35f7ffa04d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.369361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.369361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.369687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.369892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] PATH_INFO: `/v3/projects/0550987a1b1344e4b780627dc2c16214` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.371720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0550987a1b1344e4b780627dc2c16214)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.374028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.381058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.381562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.381861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.382237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] PATH_INFO: `/v3/projects/78a0c7e19e054dbaa070726b0ea08190/users/3487cf852ca841b1b59b5d7d189506f4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.449250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a0c7e19e054dbaa070726b0ea08190, user_id=3487cf852ca841b1b59b5d7d189506f4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.452690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.458954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:22.491255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 831/1661] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:22 2022] PUT /identity/v3/projects/78a0c7e19e054dbaa070726b0ea08190/users/3487cf852ca841b1b59b5d7d189506f4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:22.495108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.532173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0550987a1b1344e4b780627dc2c16214', 'request_id': 'req-b7b1a1d3-ec90-44dc-93d7-cdf483401847'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:22.551783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0550987a1b1344e4b780627dc2c16214', 'request_id': 'req-b7b1a1d3-ec90-44dc-93d7-cdf483401847'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:22.561649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0550987a1b1344e4b780627dc2c16214', 'request_id': 'req-b7b1a1d3-ec90-44dc-93d7-cdf483401847'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:22.585886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] The token cache is being invalidate because project 0550987a1b1344e4b780627dc2c16214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:22.586263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:22.589187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b7b1a1d3-ec90-44dc-93d7-cdf483401847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:22.597752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 831/1662] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:22 2022] DELETE /identity/v3/projects/0550987a1b1344e4b780627dc2c16214 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:22.607397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e98ff-37cf-4f2a-832e-7b9f731b138e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.607721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e98ff-37cf-4f2a-832e-7b9f731b138e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.607954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e98ff-37cf-4f2a-832e-7b9f731b138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.676702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0f8e98ff-37cf-4f2a-832e-7b9f731b138e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:22.677939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f8e98ff-37cf-4f2a-832e-7b9f731b138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:22.690424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b800960-df4f-4dbc-baae-43b2cc40342c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.843922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.844961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.845097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.845358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.846076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.848848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.850542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 832/1663] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:22.870068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.870277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.870555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.890153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 832/1664] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:22.891872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.898678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:22.918499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:22.921584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:22.930887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8924cb-9332-449f-af50-f235fc0fd7af admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.941723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 833/1665] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:22.942668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:22.946723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.947504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.947785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.948061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] PATH_INFO: `/v3/projects/78a0c7e19e054dbaa070726b0ea08190/users/3487cf852ca841b1b59b5d7d189506f4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.951809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a0c7e19e054dbaa070726b0ea08190, user_id=3487cf852ca841b1b59b5d7d189506f4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.960899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5d1c9-8728-4923-902e-0dc40cf74321 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:22.969484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8bcc6-d328-45ea-a237-dcd0a31be135 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:22.991236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:22.991676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:22.991952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:22.992210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:22.994657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:22.994994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:22.996957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.003433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 833/1666] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:22 2022] PUT /identity/v3/projects/78a0c7e19e054dbaa070726b0ea08190/users/3487cf852ca841b1b59b5d7d189506f4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:23.011977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.012267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.012541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.056796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 834/1667] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:23.065518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.074399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] MFA Rules not processed for user `3487cf852ca841b1b59b5d7d189506f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:23.075643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:23.098113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff03d5-e7a8-4104-9f33-9e8c25ac1deb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.211752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:23.255481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 834/1668] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:23.263755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.267353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:23.285795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:23.301226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.302045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.303102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.303102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.303339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.310509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.376240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9222e-7b85-4798-b927-13dc4ebc7330 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.386298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 835/1669] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:23 2022] POST /identity/v3/projects => generated 365 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:23.387735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.388379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.389483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.390048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.390746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.413408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.417488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.474873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 835/1670] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:23 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:23.648888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674dc3c-0b87-4a10-bf7a-fed5a639cfee admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.664919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.665555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.665838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.666114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.666764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.668834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.741146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.753066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 836/1671] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:23 2022] POST /identity/v3/users => generated 283 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:23.759129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75058f-5897-4134-9648-94f13b7713ef service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.784645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.785375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.786145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.786609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.796176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.798811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.823296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.830644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 836/1672] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:23.844428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b249de4-7f9d-4a92-89e9-7a6c3106824b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.870562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.871584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.872059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.872572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.873432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.875574 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fa450-8706-4f7c-a8d1-3f96a9167ace admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:23.897251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 837/1673] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:23.902662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:23.932314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d89323-8fa0-480e-b09d-5a2301090213 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:23.951895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:23.953068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:23.953565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:23.954051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] PATH_INFO: `/v3/projects/c2671a33610249c3abae886438b1d122/users/1c97be1778024f4aa8f9dbe87dd9db85/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:23.994499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2671a33610249c3abae886438b1d122, user_id=1c97be1778024f4aa8f9dbe87dd9db85, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:23.999963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:24.052048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.052339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.052641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.053955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 837/1674] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:23 2022] PUT /identity/v3/projects/c2671a33610249c3abae886438b1d122/users/1c97be1778024f4aa8f9dbe87dd9db85/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:24.066592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.130240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] MFA Rules not processed for user `1c97be1778024f4aa8f9dbe87dd9db85`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:24.131663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:24.231561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee2fca-5b4e-421a-b17b-73c10ec687e9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:24.242088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:24.244340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.244849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.245227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] PATH_INFO: `/v3/users/19224c3ba9564f44be82ef2d8653b161` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.247471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19224c3ba9564f44be82ef2d8653b161)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:24.251533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:24.315510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 838/1675] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:24.318944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.349943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150efa8-64b3-4fd4-b47a-fc7a5c6d2834 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:24.350968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19224c3ba9564f44be82ef2d8653b161'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:24.352536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19224c3ba9564f44be82ef2d8653b161'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:24.364342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:24.365714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.371466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19224c3ba9564f44be82ef2d8653b161'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:24.373152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.375215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.377627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:24.388821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:24.403850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19224c3ba9564f44be82ef2d8653b161'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:24.405445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc6854e6-5be9-4194-a9b8-c2849fbe31b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19224c3ba9564f44be82ef2d8653b161'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:24.451600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:24.466159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 838/1676] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:24 2022] DELETE /identity/v3/users/19224c3ba9564f44be82ef2d8653b161 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:24.475224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.476065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.477955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.486179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 839/1677] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:24 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:24.487531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.491175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:24.518858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:24.520126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:24.524913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:24.708909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9ff9-26fa-47d3-843a-e22f55b8d2c9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:24.728697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:24.733565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.733565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.733843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.734512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:24.737030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:24.769251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 839/1678] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:24.806273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 840/1679] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:24 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:24.809058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.810848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:24.820745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.862124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c16dc0-baa0-4971-a63c-1ec1433ab362 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:24.880882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:24.884202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.884202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.886654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:24.890677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:24.893944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:24.929171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 841/1680] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:24.930316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:24.957606 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed863-78a9-45eb-83ec-ab158301a7b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:24.974515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:24.976750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:24.978676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:24.978793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] PATH_INFO: `/v3/projects/18cf8f39cd65417ea253d12191a699bb/users/a8e0427befdd48e593a5a51ec79afa8c/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.001827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436069-ae94-4a18-b26f-e70bd5b8ed78 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:25.025523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:25.028024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.028310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.028581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.052443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf8f39cd65417ea253d12191a699bb, user_id=a8e0427befdd48e593a5a51ec79afa8c, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:25.063087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:25.069150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:25.070616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:25.107612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 842/1681] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:24 2022] PUT /identity/v3/projects/18cf8f39cd65417ea253d12191a699bb/users/a8e0427befdd48e593a5a51ec79afa8c/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:25.111877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:25.124097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 840/1682] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:25.288632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84ffd0-3ff4-4e00-9927-f6a5b908d838 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:25.296485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:25.297178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.297944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.298378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.299305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:25.311785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:25.333792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 843/1683] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:25.337779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:25.356275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc149-90e3-484c-bb91-d4e9472845e1 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:25.398185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:25.401168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.401461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.402027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] PATH_INFO: `/v3/projects/18cf8f39cd65417ea253d12191a699bb/users/a8e0427befdd48e593a5a51ec79afa8c/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.407384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf8f39cd65417ea253d12191a699bb, user_id=a8e0427befdd48e593a5a51ec79afa8c, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:25.420823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985705d5-19fd-4830-bcbe-32ade26f9423 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:25.435015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:25.436907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:25.464971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda9e1e-47cb-4bb7-900e-29a43070724d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:25.466338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 841/1684] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:25 2022] PUT /identity/v3/projects/18cf8f39cd65417ea253d12191a699bb/users/a8e0427befdd48e593a5a51ec79afa8c/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:25.493704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.493901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.494181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.543517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] MFA Rules not processed for user `a8e0427befdd48e593a5a51ec79afa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:25.544965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:25.641774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:25.642594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.642973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.644721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] PATH_INFO: `/v3/projects/985d43160bda4a45a77225f3387adb2f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.647785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=985d43160bda4a45a77225f3387adb2f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:25.649935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:25.666597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:25.749650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 842/1685] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:25.757726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:25.759178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:25.766992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985d43160bda4a45a77225f3387adb2f', 'request_id': 'req-05572583-8fcd-402c-aa15-5cdc7f8de4bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:25.785049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:25.787339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985d43160bda4a45a77225f3387adb2f', 'request_id': 'req-05572583-8fcd-402c-aa15-5cdc7f8de4bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:25.803712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985d43160bda4a45a77225f3387adb2f', 'request_id': 'req-05572583-8fcd-402c-aa15-5cdc7f8de4bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:25.826177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] The token cache is being invalidate because project 985d43160bda4a45a77225f3387adb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:25.826644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:25.827549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-05572583-8fcd-402c-aa15-5cdc7f8de4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:25.832517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 844/1686] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:25 2022] DELETE /identity/v3/projects/985d43160bda4a45a77225f3387adb2f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:25.836950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8373-da5b-4042-8fab-81bb87cabe46 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:25.837810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8373-da5b-4042-8fab-81bb87cabe46 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:25.837810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8373-da5b-4042-8fab-81bb87cabe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:25.880151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b81b8373-da5b-4042-8fab-81bb87cabe46 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:25.881996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b81b8373-da5b-4042-8fab-81bb87cabe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:25.888429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b07b7-234a-42b9-b5fa-4c85053c577d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.002888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 845/1687] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:26.010053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548065-9707-4799-99b2-b159645f364b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.010419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548065-9707-4799-99b2-b159645f364b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.010801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548065-9707-4799-99b2-b159645f364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.048420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-05548065-9707-4799-99b2-b159645f364b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:26.049843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-05548065-9707-4799-99b2-b159645f364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.055005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.055956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.056247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.056525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.059582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.071171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 846/1688] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:26.072558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548065-9707-4799-99b2-b159645f364b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.088828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548065-9707-4799-99b2-b159645f364b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.106263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.107241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.107859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.108269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.109104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:26.109537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.111638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.150650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 847/1689] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:26.151867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.189080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c4cb4-5436-4437-abf7-8e57a67bf065 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.203313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.203980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.204261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.204533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.205365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.210702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.212852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.224444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.248143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 843/1690] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:26.315977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 848/1691] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:26 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:26.316862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.369644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a467c-8663-40aa-a7c5-328b80872a2a service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.379208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.381264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.554097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:26.563322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.564299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.564803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.565239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.566138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.568534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.582400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ccdd-05e7-4bdf-8eee-f17666a29742 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.600909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.602083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.602686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.607693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] PATH_INFO: `/v3/users/85aba434ffdc44a191f55c4881f13327` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.609796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85aba434ffdc44a191f55c4881f13327)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.617908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 844/1692] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:26 2022] POST /identity/v3/users => generated 289 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:26.617908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.619254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.619755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.634994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:26.728032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f867a73-6cc3-4485-83dc-4d83e771ecb3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.732737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.733857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.734247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.734612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.735256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85aba434ffdc44a191f55c4881f13327'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:26.738565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.771594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85aba434ffdc44a191f55c4881f13327'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:26.788115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85aba434ffdc44a191f55c4881f13327'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:26.789137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85aba434ffdc44a191f55c4881f13327'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:26.789921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85aba434ffdc44a191f55c4881f13327'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:26.812538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 849/1693] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:26 2022] DELETE /identity/v3/users/85aba434ffdc44a191f55c4881f13327 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:26.816278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.817680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:26.868952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.872169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:26.899958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 845/1694] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:26.903068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:26.920426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7aa9-d4c4-41d4-a4d3-272f950befa9 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.969138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dce3a-87ea-4bc7-951e-7505f189203d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:26.975718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:26.976603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:26.977030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:26.977437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] PATH_INFO: `/v3/projects/31535257f93f4e32bff3b4f045fa55f4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:26.979929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] RBAC: Authorizing `identity:delete_project(project_id=31535257f93f4e32bff3b4f045fa55f4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:26.981995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.069378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:27.072225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.072519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.072796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.075821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:27.081256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67dd62-f6fe-4a25-aa82-729ea9fcbc41 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.085893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31535257f93f4e32bff3b4f045fa55f4', 'request_id': 'req-f5e2191c-7498-4ef8-919b-9107def0234a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.104150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 846/1695] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:27.105380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.105707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.106593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.108225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31535257f93f4e32bff3b4f045fa55f4', 'request_id': 'req-f5e2191c-7498-4ef8-919b-9107def0234a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.115848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31535257f93f4e32bff3b4f045fa55f4', 'request_id': 'req-f5e2191c-7498-4ef8-919b-9107def0234a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.146185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:27.151666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] The token cache is being invalidate because project 31535257f93f4e32bff3b4f045fa55f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:27.151666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.152170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.158401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 850/1696] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:26 2022] DELETE /identity/v3/projects/31535257f93f4e32bff3b4f045fa55f4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:27.159447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:27.162160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.188139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:27.196207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.342742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2191c-7498-4ef8-919b-9107def0234a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:27.352111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:27.353061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.353475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.353911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.356101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:27.358923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.377051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:27.393167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 851/1697] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:27.393167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:27.404461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.418200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 847/1698] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:27.423970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:27.429126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2b19a-96ea-4b2a-b745-ac97c2958ea5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:27.439211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:27.440330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.440614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.440888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] PATH_INFO: `/v3/users/3487cf852ca841b1b59b5d7d189506f4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.442908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3487cf852ca841b1b59b5d7d189506f4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:27.444838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.461262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6143-41ec-4d61-ac52-e4f28b5c1b07 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:27.471835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:27.475697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.478409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.478974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] PATH_INFO: `/v3/projects/a96083b38cc744588699cc530ba7a917/users/cd25f67637b94fd9a43f40a8f7b2d6b9/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.515344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96083b38cc744588699cc530ba7a917, user_id=cd25f67637b94fd9a43f40a8f7b2d6b9, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:27.522414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f4d09-1c20-46f2-9c44-0cf8128b3371 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.531388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3487cf852ca841b1b59b5d7d189506f4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.548282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 848/1699] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:27 2022] PUT /identity/v3/projects/a96083b38cc744588699cc530ba7a917/users/cd25f67637b94fd9a43f40a8f7b2d6b9/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:27.551555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0aba2-cf37-4d0e-a289-48ccc1f3b963 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.553627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0aba2-cf37-4d0e-a289-48ccc1f3b963 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.554076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0aba2-cf37-4d0e-a289-48ccc1f3b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.560477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3487cf852ca841b1b59b5d7d189506f4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.572322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3487cf852ca841b1b59b5d7d189506f4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.574416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3487cf852ca841b1b59b5d7d189506f4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.575332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3487cf852ca841b1b59b5d7d189506f4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:27.595402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 852/1700] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:27 2022] DELETE /identity/v3/users/3487cf852ca841b1b59b5d7d189506f4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:27.599908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:27.601713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.617750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c8e0aba2-cf37-4d0e-a289-48ccc1f3b963 None None] MFA Rules not processed for user `7103b8b46630401889a8e040c0e9ad62`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:27.619514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c8e0aba2-cf37-4d0e-a289-48ccc1f3b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.707973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ae053-6de7-4ad4-8a7f-b81ce5c7dc2f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:27.713241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:27.715597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.716843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.717301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.723639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.772783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 849/1701] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:27.774257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.774654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.775042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.797537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:27.838912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] MFA Rules not processed for user `cd25f67637b94fd9a43f40a8f7b2d6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:27.840400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:27.944375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:27.949150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858d88b-dccc-4caa-a111-a875442a5eb7 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:27.978059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 853/1702] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:27.981600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c74e-4ea4-4eb5-8b89-22c4ce749c61 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:27.981891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c74e-4ea4-4eb5-8b89-22c4ce749c61 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:27.983862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c74e-4ea4-4eb5-8b89-22c4ce749c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:27.991552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 850/1703] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:27.993924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:27.997439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:28.028930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc12fe5-2000-41c0-b285-372d32d36344 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.051863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1795c74e-4ea4-4eb5-8b89-22c4ce749c61 None None] MFA Rules not processed for user `7103b8b46630401889a8e040c0e9ad62`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:28.052156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.054328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1795c74e-4ea4-4eb5-8b89-22c4ce749c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:28.055781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.056118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.056498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.057258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.060366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.066315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 854/1704] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:28.068134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907822da-a777-447e-aedb-c90d20f35217 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.068762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907822da-a777-447e-aedb-c90d20f35217 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.069220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907822da-a777-447e-aedb-c90d20f35217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.121499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 851/1705] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:27 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:28.122548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.129899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:28.165194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-907822da-a777-447e-aedb-c90d20f35217 None None] MFA Rules not processed for user `956b08da3ba944d382877cf266389127`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:28.166572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-907822da-a777-447e-aedb-c90d20f35217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:28.212462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:28.264384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac09d7-e0a6-464a-a2c5-97200be680fa admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.277149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.277788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.278169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.278444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.321154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 855/1706] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:28.322258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907822da-a777-447e-aedb-c90d20f35217 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.404417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.406771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.437445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 852/1707] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:28.439495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.441604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:28.476475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da30164-dfe9-41a4-a1b6-ccadf25d5588 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.492898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907822da-a777-447e-aedb-c90d20f35217 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.498794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.499601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.499771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.503575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.508306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.512394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.512701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.512974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.513855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.517039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.525250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.526946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.553009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 853/1708] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:28.577937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 856/1709] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:28 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:28.582434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.601176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e068-dcde-45c0-838d-8e10a86c4afb service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.619838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.620462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.620746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.621019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.621811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.623910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.660319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.667483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 854/1710] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:28.677499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fb067-184a-405d-9037-492475516ebc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.688742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.689548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.689836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.690084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] PATH_INFO: `/v3/projects/1a029e73ff834f7e818444ef2d4ffaac/users/8e039742883c42849d617d1707e9932f/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.732554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a029e73ff834f7e818444ef2d4ffaac, user_id=8e039742883c42849d617d1707e9932f, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.742144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.772071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 857/1711] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:28 2022] PUT /identity/v3/projects/1a029e73ff834f7e818444ef2d4ffaac/users/8e039742883c42849d617d1707e9932f/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:28.775734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:28.798302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9b8c4-bbf5-48ed-b355-d32ce0247cee admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:28.954992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:28.957195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:28.957564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:28.957840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:28.960096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:28.964225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937c409-2e4d-4fcb-82c5-d9351aa568ac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:28.992470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 855/1712] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:28.993840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:29.009612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eb110-4600-46ce-b236-9ed130b0b18d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:29.025091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:29.025831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.031585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.031974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] PATH_INFO: `/v3/projects/1a029e73ff834f7e818444ef2d4ffaac/users/8e039742883c42849d617d1707e9932f/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.048460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a029e73ff834f7e818444ef2d4ffaac, user_id=8e039742883c42849d617d1707e9932f, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:29.053464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:29.058212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:29.081734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 858/1713] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:28 2022] PUT /identity/v3/projects/1a029e73ff834f7e818444ef2d4ffaac/users/8e039742883c42849d617d1707e9932f/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:29.097136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77618d-aa71-4423-8af4-7e6a6597e46f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.097136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77618d-aa71-4423-8af4-7e6a6597e46f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.097136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77618d-aa71-4423-8af4-7e6a6597e46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.182672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5f77618d-aa71-4423-8af4-7e6a6597e46f None None] MFA Rules not processed for user `8e039742883c42849d617d1707e9932f`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:29.188897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5f77618d-aa71-4423-8af4-7e6a6597e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:29.300535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:29.408594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 856/1714] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:29.426262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:29.428309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:29.481550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b4d9-f175-4704-a07e-03fe63972216 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.481840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b4d9-f175-4704-a07e-03fe63972216 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.482118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b4d9-f175-4704-a07e-03fe63972216 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.484009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 857/1715] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 31 04:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:06:29.492718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.493002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.493431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.584541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:29.587500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:29.626641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99af818-e668-4233-98e4-8f9b5256ffb4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:29.637104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:29.642970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.644012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.645228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.752299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 858/1716] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:06:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:29.753452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:29.756104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:29.802376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:29.804027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:29.835452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 859/1717] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:29.836529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:29.842370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:29.879989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811131c8-77cd-48d9-b308-17a513635eb3 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:29.892805 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:29.893600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.893885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.894155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.918507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:29.920339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9be42c-4506-4281-8520-fe3b06500e57 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:29.944911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 860/1718] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:29.965839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa0357-0440-451f-8470-5ca028383cb0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:29.978266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:29.979771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:29.980363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:29.980925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] PATH_INFO: `/v3/projects/78a0c7e19e054dbaa070726b0ea08190` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:29.983272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a0c7e19e054dbaa070726b0ea08190)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:29.997262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:30.165251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a0c7e19e054dbaa070726b0ea08190', 'request_id': 'req-388bb837-c82f-4425-86a4-aa5e4c95107d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.187269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a0c7e19e054dbaa070726b0ea08190', 'request_id': 'req-388bb837-c82f-4425-86a4-aa5e4c95107d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.208040 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a0c7e19e054dbaa070726b0ea08190', 'request_id': 'req-388bb837-c82f-4425-86a4-aa5e4c95107d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.211804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:30.242513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] The token cache is being invalidate because project 78a0c7e19e054dbaa070726b0ea08190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:30.242796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.244171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-388bb837-c82f-4425-86a4-aa5e4c95107d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.252442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 859/1719] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:29 2022] DELETE /identity/v3/projects/78a0c7e19e054dbaa070726b0ea08190 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:30.254291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:30.254783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:30.255106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:30.314684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:30.315953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:30.538103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 861/1720] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:30.545823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:30.546357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:30.546704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:30.553462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:30.555088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:30.586276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6a786-20ee-4ffe-b840-f88b0da235f6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:30.605882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:30.610365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:30.611174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:30.611455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:30.611726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] PATH_INFO: `/v3/projects/53dee0538d6744a3983ccd114dcc6de4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:30.615806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:30.617149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53dee0538d6744a3983ccd114dcc6de4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:30.629790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:30.657960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 860/1721] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:30.659190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:30.692023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41273c85-32b7-45be-b9f8-052bb6c7bae0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:30.717575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:30.718335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:30.718618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:30.718971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:30.720038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:30.720350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:30.721994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:30.753165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:30.775365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 861/1722] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:30.786350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:30.793240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:30.813968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53dee0538d6744a3983ccd114dcc6de4', 'request_id': 'req-3fec8e1a-754c-4e9f-b224-edd7d508c292'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.833423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53dee0538d6744a3983ccd114dcc6de4', 'request_id': 'req-3fec8e1a-754c-4e9f-b224-edd7d508c292'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.842180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48c3dd-454c-433d-a123-aea6ec41a9ef admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:30.861004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53dee0538d6744a3983ccd114dcc6de4', 'request_id': 'req-3fec8e1a-754c-4e9f-b224-edd7d508c292'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.901729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] The token cache is being invalidate because project 53dee0538d6744a3983ccd114dcc6de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:30.901829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.911344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:30.918923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 862/1723] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:30 2022] DELETE /identity/v3/projects/53dee0538d6744a3983ccd114dcc6de4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:30.923400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:30.930703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.077983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.078814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.079331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.079865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] PATH_INFO: `/v3/users/1c97be1778024f4aa8f9dbe87dd9db85` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.084354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c97be1778024f4aa8f9dbe87dd9db85)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:31.091187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:31.215270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec8e1a-754c-4e9f-b224-edd7d508c292 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:31.231196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.231881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.232165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.232438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.233089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:31.237268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fdb4f-e73b-400f-adc1-d95080bb8f88 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:31.274602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c97be1778024f4aa8f9dbe87dd9db85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:31.275610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c97be1778024f4aa8f9dbe87dd9db85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:31.293523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c97be1778024f4aa8f9dbe87dd9db85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:31.311292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c97be1778024f4aa8f9dbe87dd9db85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:31.312543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c97be1778024f4aa8f9dbe87dd9db85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:31.350491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 863/1724] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:30 2022] POST /identity/v3/projects => generated 369 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:31.354692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c53348-bb66-4ed0-b45b-92ecb0382a0a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.358148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c53348-bb66-4ed0-b45b-92ecb0382a0a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.358437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c53348-bb66-4ed0-b45b-92ecb0382a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.381789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 862/1725] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:30 2022] DELETE /identity/v3/users/1c97be1778024f4aa8f9dbe87dd9db85 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:31.383539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:31.388490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.424590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-79c53348-bb66-4ed0-b45b-92ecb0382a0a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:31.426120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79c53348-bb66-4ed0-b45b-92ecb0382a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.593146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:31.624252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1f89-99a2-4437-8541-5fef914c710d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:31.643111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.643623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 864/1726] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:31.644085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.644085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.644324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.644976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:31.650239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250369f2-6bb1-49ce-bfaf-de92312e07bc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:31.654417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.654711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.654980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.704306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 863/1727] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:31 2022] POST /identity/v3/users => generated 285 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:31.704306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.704306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.704306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.731735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:31.733338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.760985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 865/1728] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:31.761191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:31.761278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:31.761355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.782542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 864/1729] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:31.786197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:31.790895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.804571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363e4d-2fb9-42a6-80df-66d54769e77c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:31.811423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.812055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.812410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.812903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.813533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:31.815410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:31.832692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 866/1730] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:31.841969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:31.868309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45dad4-06cf-463d-ab6a-4d2d305db770 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:31.883571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.887374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.887913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.888357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.890935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:31.893337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:31.895247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:31.938042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 867/1731] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:31.941280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:31.949382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.965900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdff55-b3ff-4cec-95c7-b64cf235917f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:31.977783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:31.978514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:31.978881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:31.979452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:31.981167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:31.984003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:32.005662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.007431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.049963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 865/1732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:32.050716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.063426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70920d-e4f0-4896-9e11-36a6b57cedcc service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.070341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.071080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.071433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.071770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.072545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.074480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.099651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 866/1733] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:32.099651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.124587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a966-626f-4c0b-bb40-7445e050ba93 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.143431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.144091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.144368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.144637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.145286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.151752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.185517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4d576-4e33-41e1-b6d6-3c6fd3a2b8b8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.201656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.207981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.208225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.208403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.240427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.241899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.246822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 867/1734] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:32 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:32.247929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.287417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 868/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:32.288396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.320975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8906-5360-40fc-8000-58f0a9a1f868 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.404858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bae5ae-d8b4-4f8f-ac86-32df859ad237 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.417652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.431146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.431858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.433025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] PATH_INFO: `/v3/projects/d48f217b3350419b9f7e64c9c2ad55a2/users/9a0abf4da6e741389397a6d9a1aa3f15/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.479554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48f217b3350419b9f7e64c9c2ad55a2, user_id=9a0abf4da6e741389397a6d9a1aa3f15, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.484502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598b656-8748-41fa-bfd7-49ff1635f93f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.533012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.533233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.533466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.533736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.534374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.536346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.548764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 868/1736] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:32 2022] PUT /identity/v3/projects/d48f217b3350419b9f7e64c9c2ad55a2/users/9a0abf4da6e741389397a6d9a1aa3f15/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:32.552667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.553397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.553397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.583714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 869/1737] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:32 2022] POST /identity/v3/users => generated 286 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:32.588054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.626553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171f79-89b6-4d6c-9970-73d17cb1a373 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.709000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] MFA Rules not processed for user `9a0abf4da6e741389397a6d9a1aa3f15`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:32.713403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:32.738175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:32.796785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.797457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.797457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.797904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.798272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.802357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.827215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 870/1738] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:32.832935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.859076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2104-f0bf-4f1a-8551-59e25de53e1c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.870840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.871966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.872503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.872965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:32.899794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:32.904916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:32.940853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 871/1739] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:32.941920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:32.962482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dcf2-1483-4587-9515-34284167c205 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:32.979745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:32.980561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:32.981295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:32.981901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] PATH_INFO: `/v3/projects/16dda3204e1e47b88fe328c0f8df2e02/users/15d8abfc779d4aeca4d933edd016d3ca/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.009458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] RBAC: Authorizing `identity:create_grant(project_id=16dda3204e1e47b88fe328c0f8df2e02, user_id=15d8abfc779d4aeca4d933edd016d3ca, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.013858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc3171-7a87-4649-9b4e-912dbcb3bfab admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.019786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 869/1740] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:33.031215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.051384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a5fc-087c-4401-b3cb-70c578d559fd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.058714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.060376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.061266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.061737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.062576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.064951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.107117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 872/1741] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:32 2022] PUT /identity/v3/projects/16dda3204e1e47b88fe328c0f8df2e02/users/15d8abfc779d4aeca4d933edd016d3ca/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:33.108736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.109021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.109291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.135105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:33.137966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 870/1742] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:33 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:33.142336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.182482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] MFA Rules not processed for user `15d8abfc779d4aeca4d933edd016d3ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:33.183832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:33.336371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721bd19-0c71-43c3-8c2c-6e6103e61a0b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.347743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.348370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.348636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.349010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.349528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.352329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.415680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 873/1743] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:33.419956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.437551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 871/1744] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:33 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:33.442835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.454464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18481a-3d67-41c6-91aa-7a904c78fd05 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.470248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca971-b3a1-4069-8e32-d5fbfbb61d09 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.474719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.475452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.475735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.476040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.476695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.478779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.492127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.492758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.493111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.493399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.494143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.497080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.526475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 872/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:33.527929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.557735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 874/1746] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:33 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:33.568998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.572523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a56c9-e0ed-488a-9f2c-ba587b261f2e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.786121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0bd50-c147-482f-8edc-1c26125534ea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.794397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.797219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.798069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.798592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] PATH_INFO: `/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0/users/f35038054c6d475d90de799cff447536/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.801312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:33.802514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:33.803005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:33.803891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:33.804973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:33.807713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.812338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.852017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea788ab7bfa34c08b286c932a2bc91e0, user_id=f35038054c6d475d90de799cff447536, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:33.856829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:33.873981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 875/1747] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:33 2022] POST /identity/v3/users => generated 292 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:33.875601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:33.894934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 873/1748] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:33 2022] PUT /identity/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0/users/f35038054c6d475d90de799cff447536/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:33.896487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454c97-53cb-4e69-8b93-c95e2e7fb59b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:33.899271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.018481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbe580-6fa9-44c9-af14-c056c2cb07b1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.025549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.026720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.027717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.028314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.029775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.032725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.049502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 874/1749] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:34.056460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.072129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947f954-13dc-414e-a447-9dd0dc341fb7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.074149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.074873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.075302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.075605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.076281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.077993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.080260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.081420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.081893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.082390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] PATH_INFO: `/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0/users/f35038054c6d475d90de799cff447536/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.085715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea788ab7bfa34c08b286c932a2bc91e0, user_id=f35038054c6d475d90de799cff447536, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.089308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfac00-25dc-498a-bd05-2db4d4732450 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.093579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:34.114157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 876/1750] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:34.118612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.133558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 875/1751] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:34 2022] PUT /identity/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0/users/f35038054c6d475d90de799cff447536/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:34.134983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.135477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.135696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.153028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4e128-226d-402c-acd3-49f19ddac4af admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.207352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] MFA Rules not processed for user `f35038054c6d475d90de799cff447536`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:34.216251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:34.272510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.273748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.274456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.274931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] PATH_INFO: `/v3/projects/edcb5eb0fbf34555be2102e4245cb35b/users/6a4ebd0a645c47e8b7205ff3aaa7dab6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.301303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcb5eb0fbf34555be2102e4245cb35b, user_id=6a4ebd0a645c47e8b7205ff3aaa7dab6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.305296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.325228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 877/1752] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:34 2022] PUT /identity/v3/projects/edcb5eb0fbf34555be2102e4245cb35b/users/6a4ebd0a645c47e8b7205ff3aaa7dab6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:34.328968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.402808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 876/1753] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:34.430231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.435780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:34.539850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe8d03-0995-48c8-9c35-7ac0c0b972b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.555146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.556044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.556409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.556760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.557509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.559459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.585638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 878/1754] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:34.595334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.609040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b72e-61bc-4fdb-a8d9-26261867eece admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.613120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b099-7e14-4591-ba68-0c3e4bd99091 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.621242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:34.624081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.625401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.626085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.626471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] PATH_INFO: `/v3/projects/edcb5eb0fbf34555be2102e4245cb35b/users/6a4ebd0a645c47e8b7205ff3aaa7dab6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.633116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcb5eb0fbf34555be2102e4245cb35b, user_id=6a4ebd0a645c47e8b7205ff3aaa7dab6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.636003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf4899-17d8-4d65-a556-8a24ca91eb35 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.669620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 879/1755] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:34 2022] PUT /identity/v3/projects/edcb5eb0fbf34555be2102e4245cb35b/users/6a4ebd0a645c47e8b7205ff3aaa7dab6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:34.681633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.683796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.687180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.698338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:34.698964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:34.699298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:34.699564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:34.749197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] MFA Rules not processed for user `6a4ebd0a645c47e8b7205ff3aaa7dab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:34.752068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:34.819252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:34.826316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:34.827964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:34.841424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 877/1756] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:34.900956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 880/1757] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:34.917856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.920615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:34.963928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b217df-7d79-4276-857f-f91a1348c7aa service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:34.991439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:34.992939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:35.110941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:35.111603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.111883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.112150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.160953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:35.163830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911e380-7cc3-43ee-9ebd-aeb3d0c61f1a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:35.164556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f027a8-9a52-4296-bd74-49f98233ce50 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:35.184131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:35.185229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.186814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.187214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] PATH_INFO: `/v3/projects/c2671a33610249c3abae886438b1d122` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.191377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2671a33610249c3abae886438b1d122)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:35.196864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:35.202207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 878/1758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:35.203700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.203935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.206923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.284244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] MFA Rules not processed for user `f80c8c1d6af7450eb8f936df231ad63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:35.285621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:35.385610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2671a33610249c3abae886438b1d122', 'request_id': 'req-5d8f97a2-0126-46a1-8058-5f20fa37f642'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.402089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2671a33610249c3abae886438b1d122', 'request_id': 'req-5d8f97a2-0126-46a1-8058-5f20fa37f642'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.418729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2671a33610249c3abae886438b1d122', 'request_id': 'req-5d8f97a2-0126-46a1-8058-5f20fa37f642'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.425277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:35.443392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] The token cache is being invalidate because project c2671a33610249c3abae886438b1d122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:35.444010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.444982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.448017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 881/1759] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:34 2022] DELETE /identity/v3/projects/c2671a33610249c3abae886438b1d122 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:35.456840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:35.458762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:35.474744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 879/1760] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:35.475770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:35.513222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbecd2f-e363-4f09-941e-7072bec94957 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:35.607728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f97a2-0126-46a1-8058-5f20fa37f642 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:35.625664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:35.628499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.628921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.629298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.632684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:35.652791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:35.655103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.655103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.655324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] PATH_INFO: `/v3/users/a8e0427befdd48e593a5a51ec79afa8c` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.658716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e0427befdd48e593a5a51ec79afa8c)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:35.662165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:35.717742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:35.784684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:35.786372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:35.803169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e0427befdd48e593a5a51ec79afa8c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.803989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e0427befdd48e593a5a51ec79afa8c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.828039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 882/1761] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:35.829361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:35.833232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:35.835047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e0427befdd48e593a5a51ec79afa8c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.855967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e0427befdd48e593a5a51ec79afa8c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.856868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bceeb9e3-3135-4b17-8a89-c2cc2d5b3dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e0427befdd48e593a5a51ec79afa8c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:35.898874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 880/1762] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:35 2022] DELETE /identity/v3/users/a8e0427befdd48e593a5a51ec79afa8c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:35.901852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f966278-8696-4f4c-a71a-37cc6bb549a7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:35.903314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f966278-8696-4f4c-a71a-37cc6bb549a7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:35.905798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f966278-8696-4f4c-a71a-37cc6bb549a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:35.952600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0f966278-8696-4f4c-a71a-37cc6bb549a7 None None] MFA Rules not processed for user `77fb76da34bf44a1b214a08bb4b8e51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:35.954941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0f966278-8696-4f4c-a71a-37cc6bb549a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.077471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf99b0-a8da-46ff-bbfc-40bcbfddd1ed service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:36.098904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:36.109847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 881/1763] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:36.120978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:36.121432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:36.121818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:36.154465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] MFA Rules not processed for user `956b08da3ba944d382877cf266389127`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:36.156475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.256289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 882/1764] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:36.256289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:36.277161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.338001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:36.338257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:36.338536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:36.338802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:36.346124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.473369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df74683-2e1b-4bed-915e-72cc3809c2cb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:36.495482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:36.496120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:36.496403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:36.496677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:36.501881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:36.502079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.507887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884584b4-3c37-4b5e-9eb5-337cc48a5fac service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:36.519751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 883/1765] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:36.536400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:36.536692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:36.545391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:36.629493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] MFA Rules not processed for user `77fb76da34bf44a1b214a08bb4b8e51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:36.633974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.663792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 884/1766] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:36.752140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:36.753795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:36.778285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:36.796695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110ed57-c2ae-49c9-869b-1e90f9492b4e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:36.799835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:36.807568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:36.820313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:36.821301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:36.822592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:36.822882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:36.851138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 883/1767] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:36.855488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:36.858119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6bac8-769f-40fc-8d58-d735167611f8 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:36.926709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 885/1768] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:37.088245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:37.090021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:37.213224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8e-548d-4d15-93fe-c58c3eb13d6f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:37.213613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8e-548d-4d15-93fe-c58c3eb13d6f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:37.214500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8e-548d-4d15-93fe-c58c3eb13d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:37.299776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c5b96a8e-548d-4d15-93fe-c58c3eb13d6f None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:37.301285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c5b96a8e-548d-4d15-93fe-c58c3eb13d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:37.323181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:37.338942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5f8d-d933-43dc-8e4d-079f10dbf58f service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:37.352280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:37.352942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:37.353219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:37.353494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] PATH_INFO: `/v3/projects/18cf8f39cd65417ea253d12191a699bb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:37.360421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cf8f39cd65417ea253d12191a699bb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:37.367267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:37.560047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 886/1769] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:37.591439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cf8f39cd65417ea253d12191a699bb', 'request_id': 'req-43efdee2-60d8-4d4f-b4c7-ca569eb75738'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:37.624922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cf8f39cd65417ea253d12191a699bb', 'request_id': 'req-43efdee2-60d8-4d4f-b4c7-ca569eb75738'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:37.634447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cf8f39cd65417ea253d12191a699bb', 'request_id': 'req-43efdee2-60d8-4d4f-b4c7-ca569eb75738'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:37.676706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] The token cache is being invalidate because project 18cf8f39cd65417ea253d12191a699bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:37.677004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:37.681380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:37.687017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 884/1770] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:37 2022] DELETE /identity/v3/projects/18cf8f39cd65417ea253d12191a699bb => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:37.693173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746084d-c4ce-4e99-9b9e-a5a9bbdde641 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:37.694030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746084d-c4ce-4e99-9b9e-a5a9bbdde641 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:37.695122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746084d-c4ce-4e99-9b9e-a5a9bbdde641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:37.738383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b746084d-c4ce-4e99-9b9e-a5a9bbdde641 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:37.740198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b746084d-c4ce-4e99-9b9e-a5a9bbdde641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:37.880003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:37.881810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:37.989935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 887/1771] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:37.999274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:37.999611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:37.999940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.067775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:38.072188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:38.104082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 888/1772] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:38.107945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:38.138656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0e32a-93a0-435c-9e48-c4d8adcb3e6c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:38.172629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:38.173259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:38.173562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:38.173806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.174467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:38.174878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:38.181865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:38.217722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 889/1773] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:38.219689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:38.242630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00173834-b41d-43da-8d2f-b3bb0aed7c2d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:38.252912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efdee2-60d8-4d4f-b4c7-ca569eb75738 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:38.275998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:38.277556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:38.277893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:38.278148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.283787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:38.289866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:38.290893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:38.291369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:38.291492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:38.292421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.298429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:38.373430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 890/1774] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:38 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:38.378795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:38.505576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:38.540384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:38.542807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:38.585392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 885/1775] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:06:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:06:38.751500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce72ab3-328a-40e8-b9ab-939219035782 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:38.773202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:38.776340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:38.778509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:38.779261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.784037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:38.796361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:38.911312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:38.918946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 891/1776] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:38 2022] POST /identity/v3/users => generated 285 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:38.933253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e107c-943a-4931-921f-7b3661ad0aad service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:38.981438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:38.982837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:38.986367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:38.988822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:38.990995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:38.995666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:39.001748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:39.041463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 886/1777] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:39.043607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:39.069558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b5bc2-ab79-4993-b9fe-fd5f3b6b49eb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:39.087353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:39.088062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:39.088413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:39.088774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:39.089520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:39.091847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb70e8-d32c-4123-a0f6-f777b3bd2ab9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:39.119686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 892/1778] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:39.120901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:39.144554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72d542-3453-4bfb-965a-89dc8d89a5cc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:39.170578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:39.172334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:39.174937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:39.175597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] PATH_INFO: `/v3/projects/74b87deb41334c5195849a7852bccbf9/users/1b289bfd11d44d879ce2c3d5f15ab0e8/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:39.257631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b87deb41334c5195849a7852bccbf9, user_id=1b289bfd11d44d879ce2c3d5f15ab0e8, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:39.273653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:39.304781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 887/1779] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:39 2022] PUT /identity/v3/projects/74b87deb41334c5195849a7852bccbf9/users/1b289bfd11d44d879ce2c3d5f15ab0e8/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:39.316649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d960f-0921-4991-bec8-a94695575de2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:39.316994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d960f-0921-4991-bec8-a94695575de2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:39.317611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d960f-0921-4991-bec8-a94695575de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:39.408017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-266d960f-0921-4991-bec8-a94695575de2 None None] MFA Rules not processed for user `1b289bfd11d44d879ce2c3d5f15ab0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:39.413551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-266d960f-0921-4991-bec8-a94695575de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:39.709105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 893/1780] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:39.711588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:39.975912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62fe0c-256c-4802-a681-214e6bd46b56 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:39.989363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:39.990515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:39.992484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:39.994423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:39.997432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:40.003886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d422-aed5-4d6e-bcd4-5867576f8093 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:40.102291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 888/1781] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:39 2022] POST /identity/v3/projects => generated 369 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:40.106227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d960f-0921-4991-bec8-a94695575de2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:40.131523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d960f-0921-4991-bec8-a94695575de2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:40.178903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50388ccf-46b2-4f16-b519-8e33515b03f6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.179631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50388ccf-46b2-4f16-b519-8e33515b03f6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.180223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50388ccf-46b2-4f16-b519-8e33515b03f6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.182497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 889/1782] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 31 04:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:06:40.190872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.191434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.191856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.306582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:40.308382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:40.417856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:40.423400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.424173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.424763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.425678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:40.430901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:40.520649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 894/1783] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:40 2022] POST /identity/v3/users => generated 291 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:40.522764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:40.527813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:40.566900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3edccd-dfb0-4241-81be-c6bc9860bd28 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:40.568191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 890/1784] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:06:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:40.569217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:40.582458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:40.583514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.583963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.584388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] PATH_INFO: `/v3/users/9a0abf4da6e741389397a6d9a1aa3f15` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.585727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0abf4da6e741389397a6d9a1aa3f15)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:40.587894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:40.617747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8a70-89db-46a1-9302-4fd286b2b981 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:40.629607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:40.640894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:40.643518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.643805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.644078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.648881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:40.651494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:40.671503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 891/1785] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:40.676708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:40.685219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:40.743594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0abf4da6e741389397a6d9a1aa3f15'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:40.745040 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f50a8-7ccf-4697-9a5e-f92f4989f5f0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:40.756287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:40.757101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.758287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.758821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.779246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:40.780375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0abf4da6e741389397a6d9a1aa3f15'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:40.780932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:40.792361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0abf4da6e741389397a6d9a1aa3f15'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:40.793276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0abf4da6e741389397a6d9a1aa3f15'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:40.793924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7a7f6380-754f-40ed-b138-746c7431f1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0abf4da6e741389397a6d9a1aa3f15'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:40.808805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 892/1786] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:40.809622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:40.814189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 895/1787] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:40 2022] DELETE /identity/v3/users/9a0abf4da6e741389397a6d9a1aa3f15 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:40.815480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0729a1a-ef00-4059-a6ea-43c2f602e8f0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.815763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0729a1a-ef00-4059-a6ea-43c2f602e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.816044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0729a1a-ef00-4059-a6ea-43c2f602e8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:40.838581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d0729a1a-ef00-4059-a6ea-43c2f602e8f0 None None] MFA Rules not processed for user `f80c8c1d6af7450eb8f936df231ad63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:40.839929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d0729a1a-ef00-4059-a6ea-43c2f602e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:40.841842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbf54b-ffd2-44e1-ac19-f5356806a64c service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:40.940309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:40.976682 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 896/1788] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:40.998504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d46657-126e-43af-997c-71480545b7e6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:40.998504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d46657-126e-43af-997c-71480545b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:40.998504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d46657-126e-43af-997c-71480545b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.030108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:41.031514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:41.032188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:41.032669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] PATH_INFO: `/v3/projects/16f35b9b73fe49c6a6f4168507dae294/users/85f3920da4bd4b819e7f9eed07700ca5/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.059514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f35b9b73fe49c6a6f4168507dae294, user_id=85f3920da4bd4b819e7f9eed07700ca5, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:41.062550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:41.082175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 893/1789] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:40 2022] PUT /identity/v3/projects/16f35b9b73fe49c6a6f4168507dae294/users/85f3920da4bd4b819e7f9eed07700ca5/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:41.086835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:41.091095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-45d46657-126e-43af-997c-71480545b7e6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:41.092427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:41.103626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45d46657-126e-43af-997c-71480545b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:41.302696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 897/1790] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:41.310615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf4864-7d52-462e-ba90-9f382dbce360 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:41.314273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d46657-126e-43af-997c-71480545b7e6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:41.315891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45d46657-126e-43af-997c-71480545b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:41.337094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:41.338799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:41.339315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:41.339797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.341072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:41.344834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:41.392121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 894/1791] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:41.402367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:41.438240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ed-4d16-4ff4-a25e-56823a9dcbc2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:41.466676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:41.471269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:41.471974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:41.475811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] PATH_INFO: `/v3/projects/16f35b9b73fe49c6a6f4168507dae294/users/85f3920da4bd4b819e7f9eed07700ca5/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.481466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f35b9b73fe49c6a6f4168507dae294, user_id=85f3920da4bd4b819e7f9eed07700ca5, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:41.495925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde929-d318-4338-a509-85b1ff0744e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:41.499411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:41.531250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d46657-126e-43af-997c-71480545b7e6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:41.583471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 895/1792] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:41 2022] PUT /identity/v3/projects/16f35b9b73fe49c6a6f4168507dae294/users/85f3920da4bd4b819e7f9eed07700ca5/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:41.594815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c343-1733-4b7a-9c1b-e73eb5a546ac None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:41.601477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c343-1733-4b7a-9c1b-e73eb5a546ac None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:41.601810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c343-1733-4b7a-9c1b-e73eb5a546ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.702411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:41.704813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:41.707445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:41.707820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:41.790678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c5c3c343-1733-4b7a-9c1b-e73eb5a546ac None None] MFA Rules not processed for user `85f3920da4bd4b819e7f9eed07700ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:41.793002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c5c3c343-1733-4b7a-9c1b-e73eb5a546ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:41.999979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:42.002282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:42.062318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 896/1793] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:42.064019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:42.064360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:42.064570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:42.072452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 898/1794] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:42.074619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:42.077947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.123949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] MFA Rules not processed for user `77fb76da34bf44a1b214a08bb4b8e51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:42.126483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.249980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:42.275538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6028105-a4ac-4886-83dc-9c9b05c6d0a9 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:42.287717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:42.288725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:42.289191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:42.289557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:42.331108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:42.333367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b394e-db80-4842-820e-d39fd5fc3c2b service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:42.361690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 899/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:42.415397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:42.415746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:42.416164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:42.420111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 897/1796] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:42.473363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:42.476722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.481467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:42.483859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.500495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 900/1797] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Aug 31 04:06:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:42.502525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c760b9b-ecb6-4260-9929-077b5650d68d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:42.534655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:42.542645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:42.542828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:42.543199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:42.552179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.770762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:42.776035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:42.808476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 898/1798] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:42.869945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:42.871640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:42.902654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58fab0-a0ef-4a62-8ab0-27530aab54f0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:42.916504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:42.918133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:42.918486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:42.918824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] PATH_INFO: `/v3/projects/d48f217b3350419b9f7e64c9c2ad55a2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:42.922565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48f217b3350419b9f7e64c9c2ad55a2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:42.927204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:43.036577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:43.079937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48f217b3350419b9f7e64c9c2ad55a2', 'request_id': 'req-1d7452ee-5531-4de8-a438-43d0d46337e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.100306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48f217b3350419b9f7e64c9c2ad55a2', 'request_id': 'req-1d7452ee-5531-4de8-a438-43d0d46337e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.137273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48f217b3350419b9f7e64c9c2ad55a2', 'request_id': 'req-1d7452ee-5531-4de8-a438-43d0d46337e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.170251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] The token cache is being invalidate because project d48f217b3350419b9f7e64c9c2ad55a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:43.172694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.174550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.179516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 901/1799] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:42 2022] DELETE /identity/v3/projects/d48f217b3350419b9f7e64c9c2ad55a2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:43.183934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:43.207742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0c53f-ccb0-4659-b7f5-fd79b3127c43 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:43.381893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:43.382752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:43.383177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:43.383457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] PATH_INFO: `/v3/users/f35038054c6d475d90de799cff447536` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:43.384710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35038054c6d475d90de799cff447536)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:43.386945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:43.536485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35038054c6d475d90de799cff447536'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.540174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35038054c6d475d90de799cff447536'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.580801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35038054c6d475d90de799cff447536'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.583629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:43.585097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:43.610456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35038054c6d475d90de799cff447536'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.611618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35038054c6d475d90de799cff447536'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:43.675279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:43.697218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 899/1800] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:43 2022] DELETE /identity/v3/users/f35038054c6d475d90de799cff447536 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:43.740407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7452ee-5531-4de8-a438-43d0d46337e4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:43.876365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:43.877334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:43.877769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:43.878171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:43.883531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:43.929686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:43.932636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfbf92-3fd6-42c1-81ca-a01b3992df78 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:43.962813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 902/1801] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:44.161977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:44.163606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:44.352431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0641f-1943-4c10-87bd-afea7180ef45 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:44.377703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:44.384096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:44.384579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:44.384745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] PATH_INFO: `/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:44.428555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea788ab7bfa34c08b286c932a2bc91e0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:44.438115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:44.616764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea788ab7bfa34c08b286c932a2bc91e0', 'request_id': 'req-1dc12458-e318-47a7-b5bd-fb777ef2deb7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:44.636689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea788ab7bfa34c08b286c932a2bc91e0', 'request_id': 'req-1dc12458-e318-47a7-b5bd-fb777ef2deb7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:44.664337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea788ab7bfa34c08b286c932a2bc91e0', 'request_id': 'req-1dc12458-e318-47a7-b5bd-fb777ef2deb7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:44.715732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] The token cache is being invalidate because project ea788ab7bfa34c08b286c932a2bc91e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:44.716632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:44.735962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1dc12458-e318-47a7-b5bd-fb777ef2deb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:44.747548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 900/1802] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:44 2022] DELETE /identity/v3/projects/ea788ab7bfa34c08b286c932a2bc91e0 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:44.749346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:44.749991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:44.750196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:44.848229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:44.852483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:44.941866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 903/1803] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:44.950497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:44.950862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:44.951170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.003605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:45.005727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:45.023123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 901/1804] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:45.025820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.052531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b641ba6-f941-4872-9781-3f274ee498ad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.083221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.084023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.084307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.084556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.085249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:45.085566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.087308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.144927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 904/1805] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:06:45.146895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.171020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f4298-b921-4748-a8ec-9c8c9b30963a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.184282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.185055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.185380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.185681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.190617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.202197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.344497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.351643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 902/1806] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:45 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:45.360787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1752ad7-ba77-44f6-bcc7-d7ea639b185b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.375951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:45.498307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.499434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.500620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.501252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.502546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.505586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.560031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 905/1807] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:45 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:45.562735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.579364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4764-a518-4b61-8184-4e1bf6b79332 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.586665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.587723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.588725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.589073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.589867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.593021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.613391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.622334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 903/1808] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:45.634802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930bd77-48eb-4414-8899-bf91bc37f9f9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.642139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.642774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.643146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.643524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.644184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.646753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3db22b-580d-4908-b20a-5f3ec3dde0d5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.686319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:45.697313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 906/1809] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:45.699790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0b83-2562-4812-9f4f-274030c3a561 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:45.706406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:45.707286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.707674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.708064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] PATH_INFO: `/v3/projects/28d82c7df412404fac16026ea268db02/users/6d6158f1905d4a0ba3498c2dc8ffbab4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.785987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d82c7df412404fac16026ea268db02, user_id=6d6158f1905d4a0ba3498c2dc8ffbab4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:45.808116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:45.867302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:45.867965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:45.867965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:45.873875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 904/1810] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:45 2022] PUT /identity/v3/projects/28d82c7df412404fac16026ea268db02/users/6d6158f1905d4a0ba3498c2dc8ffbab4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:45.958991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] MFA Rules not processed for user `6d6158f1905d4a0ba3498c2dc8ffbab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:45.962253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:46.356727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:46.361792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 907/1811] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:46.486857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:46.636191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1615c6-75fc-4be9-9f7d-cc8100fade21 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:46.661773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:46.662405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:46.662831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:46.663277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:46.663928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:46.666003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:46.757919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 905/1812] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:46 2022] POST /identity/v3/projects => generated 379 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:46.762633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:46.792046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fa8cf-68e4-4b24-9b10-884dec92c96d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:47.101696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:47.102462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.102734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.103048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.115631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:47.118127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:47.226681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 908/1813] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:46 2022] POST /identity/v3/users => generated 296 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:47.230657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:47.264144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9247462-ec19-47ed-8fa5-432c7b3c898f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:47.330416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:47.331713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.335734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.337347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.338837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:47.343395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:47.400977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 906/1814] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:47.403974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:47.414993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8dd5c-b073-4a66-b3d6-cedf363b7a51 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:47.446939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:47.447673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.447950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.448243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] PATH_INFO: `/v3/projects/a1fb712ac4714f24bf16a5566acbae6e/users/38db97fabfbb45c28066e377a1319753/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.457338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:47.510192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fb712ac4714f24bf16a5566acbae6e, user_id=38db97fabfbb45c28066e377a1319753, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:47.513746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:47.547086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:47.547593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 909/1815] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:47 2022] PUT /identity/v3/projects/a1fb712ac4714f24bf16a5566acbae6e/users/38db97fabfbb45c28066e377a1319753/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:47.562803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b2fff-83ea-45b6-9e3b-438d9ab8b474 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:47.785274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:47.786710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.787199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.787622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.788424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:47.790365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4661c55-479f-4742-a477-22dfb1456403 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:47.840608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:47.841093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 907/1816] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:47.857491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd481f7-24c3-4bbb-9864-b13f33000203 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:47.880516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:47.881380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.887536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.888020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] PATH_INFO: `/v3/projects/a1fb712ac4714f24bf16a5566acbae6e/users/38db97fabfbb45c28066e377a1319753/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.893202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8ef1-4afb-4bac-b48e-7e599c97d22a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.893656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8ef1-4afb-4bac-b48e-7e599c97d22a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.894070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8ef1-4afb-4bac-b48e-7e599c97d22a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.895603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 908/1817] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:06:47.905849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fb712ac4714f24bf16a5566acbae6e, user_id=38db97fabfbb45c28066e377a1319753, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:47.908548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92067c-8795-4864-a565-52a2665b6d64 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:47.969450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 910/1818] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:47 2022] PUT /identity/v3/projects/a1fb712ac4714f24bf16a5566acbae6e/users/38db97fabfbb45c28066e377a1319753/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:47.978133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.978618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.979052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:47.980062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:47.980462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:47.980827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:48.033173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:48.034770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:48.082002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] MFA Rules not processed for user `38db97fabfbb45c28066e377a1319753`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:48.085111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:48.280720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 911/1819] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:48.323159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:48.324735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:48.358258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 909/1820] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:06:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:48.504346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:48.506218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:48.512134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fde9d-8f7c-41d8-99cf-c51a91586c73 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:48.523984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:48.525433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:48.525722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:48.526963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:48.531793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d164f84-c8b9-4323-9156-cb2c2de57a67 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:48.539395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:48.540494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:48.540957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:48.541432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:48.551472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:48.554465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:48.554590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:48.556952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:48.568752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 912/1821] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:48.596457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 910/1822] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:48.883262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:48.886024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:48.961060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:48.965630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bacdc-d37b-4eb3-80e3-99052fc592e7 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:48.979752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:48.979752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:48.979752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:48.979752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] PATH_INFO: `/v3/users/1b289bfd11d44d879ce2c3d5f15ab0e8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:48.980116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b289bfd11d44d879ce2c3d5f15ab0e8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:48.980116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.025174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.032669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b289bfd11d44d879ce2c3d5f15ab0e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.039211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e797-a7bc-43eb-9072-047772bdf3f9 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:49.045724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38db97fabfbb45c28066e377a1319753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1fb712ac4714f24bf16a5566acbae6e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:49.046544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.047078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.047796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.048315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b289bfd11d44d879ce2c3d5f15ab0e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.048870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:49.049315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:49.052896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.058307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b289bfd11d44d879ce2c3d5f15ab0e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.059176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b289bfd11d44d879ce2c3d5f15ab0e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.059894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b33f658a-4ce1-4f40-a754-f960336d4327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b289bfd11d44d879ce2c3d5f15ab0e8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.065049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:49.074968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 911/1823] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:06:49.077245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 913/1824] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:48 2022] DELETE /identity/v3/users/1b289bfd11d44d879ce2c3d5f15ab0e8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:49.081347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.083602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.083854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.083949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.087075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:49.106306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4343f-da6f-430e-a0ec-af874d399481 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:49.111007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:49.112666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:49.187483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38db97fabfbb45c28066e377a1319753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1fb712ac4714f24bf16a5566acbae6e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:49.188193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.188645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.188926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.189587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:49.191560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.193010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 914/1825] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:49.216277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.218728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:49.236383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 912/1826] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:49 2022] POST /identity/v3/projects => generated 355 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:49.241334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.314181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8808cb6-6022-40e7-be17-64294eaf4b8a tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:49.319265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38db97fabfbb45c28066e377a1319753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1fb712ac4714f24bf16a5566acbae6e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:49.320016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.320401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.320772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.321572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:49.321992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:49.324383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.349742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 913/1827] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:49.369702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18162c54-080c-45e8-a956-ec3e8bed2bb2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:49.395889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.414695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490555-4fdf-4c2d-9889-6f14b9d2e656 tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:49.427979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:49.428816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38db97fabfbb45c28066e377a1319753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1fb712ac4714f24bf16a5566acbae6e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:49.429696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.430294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.430773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] PATH_INFO: `/v3/projects/68ca6b6415cb4f00b428e802cb5e23ee` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.435543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorizing `identity:delete_project(project_id=68ca6b6415cb4f00b428e802cb5e23ee)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:49.437766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.455003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:49.455668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:49.456121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:49.456415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:49.531655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ca6b6415cb4f00b428e802cb5e23ee', 'request_id': 'req-a25fe298-6aea-482c-9812-bac6c68e0d4c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.562286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ca6b6415cb4f00b428e802cb5e23ee', 'request_id': 'req-a25fe298-6aea-482c-9812-bac6c68e0d4c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.583319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ca6b6415cb4f00b428e802cb5e23ee', 'request_id': 'req-a25fe298-6aea-482c-9812-bac6c68e0d4c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.591872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:49.593800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:49.598643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:49.607851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] The token cache is being invalidate because project 68ca6b6415cb4f00b428e802cb5e23ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:49.610257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.611364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:49.616954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 914/1828] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:49 2022] DELETE /identity/v3/projects/68ca6b6415cb4f00b428e802cb5e23ee => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:49.618590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 915/1829] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:49.951705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:49.954155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:50.017725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:50.019770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:50.038504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe298-6aea-482c-9812-bac6c68e0d4c tempest-VolumeQuotaClassesTest-147911920 tempest-VolumeQuotaClassesTest-147911920-project-admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:50.044696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:50.045407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:50.045612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:50.045879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] PATH_INFO: `/v3/users/6d6158f1905d4a0ba3498c2dc8ffbab4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:50.047300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6158f1905d4a0ba3498c2dc8ffbab4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:50.049203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14adfb-54aa-4efa-8fba-f0696656a281 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:50.049319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:50.056766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:50.058250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:50.058634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:50.058916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] PATH_INFO: `/v3/projects/74b87deb41334c5195849a7852bccbf9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:50.060111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b87deb41334c5195849a7852bccbf9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:50.062769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:50.106071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6158f1905d4a0ba3498c2dc8ffbab4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.106640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6158f1905d4a0ba3498c2dc8ffbab4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.113772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b87deb41334c5195849a7852bccbf9', 'request_id': 'req-69018248-ec7c-4211-9a16-ea426e11151a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.113952 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6158f1905d4a0ba3498c2dc8ffbab4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.120829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b87deb41334c5195849a7852bccbf9', 'request_id': 'req-69018248-ec7c-4211-9a16-ea426e11151a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.123426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6158f1905d4a0ba3498c2dc8ffbab4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.124074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3900fcd1-b7df-447f-9624-0caab721185a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6158f1905d4a0ba3498c2dc8ffbab4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.126820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b87deb41334c5195849a7852bccbf9', 'request_id': 'req-69018248-ec7c-4211-9a16-ea426e11151a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.138784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] The token cache is being invalidate because project 74b87deb41334c5195849a7852bccbf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:50.139123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.139749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.141631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 916/1830] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:50 2022] DELETE /identity/v3/projects/74b87deb41334c5195849a7852bccbf9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:50.142803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 915/1831] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:49 2022] DELETE /identity/v3/users/6d6158f1905d4a0ba3498c2dc8ffbab4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:50.145751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:50.146705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:50.147020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:50.147328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:50.147602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:50.166522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:50.167740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:50.213473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69018248-ec7c-4211-9a16-ea426e11151a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:50.224344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:50.229696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 916/1832] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:50.240496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:50.241939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:50.263363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:50.275442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:50.276216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:50.276491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:50.276984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] PATH_INFO: `/v3/users/85f3920da4bd4b819e7f9eed07700ca5` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:50.278092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f3920da4bd4b819e7f9eed07700ca5)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:50.280087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:50.302706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd21a4-0ad3-4429-9ae4-9d61d40b65c5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:50.307500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:50.308135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:50.308512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:50.308791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:50.319262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:50.320708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:50.327545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f3920da4bd4b819e7f9eed07700ca5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.334676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 917/1833] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:50.339976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f3920da4bd4b819e7f9eed07700ca5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.347451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f3920da4bd4b819e7f9eed07700ca5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.348183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f3920da4bd4b819e7f9eed07700ca5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.348795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f3920da4bd4b819e7f9eed07700ca5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:50.372793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 917/1834] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:50 2022] DELETE /identity/v3/users/85f3920da4bd4b819e7f9eed07700ca5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:51.165039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:51.166684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:51.189555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:51.191208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:51.227226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cdd4e-425d-443d-8a65-a5ae14ee0051 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:51.305823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953bf8e-c9c2-4087-9eda-7b15d157da07 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:51.324895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:51.325968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.326452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.326935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] PATH_INFO: `/v3/projects/16f35b9b73fe49c6a6f4168507dae294` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.328752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f35b9b73fe49c6a6f4168507dae294)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:51.334539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:51.411835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:51.413077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.416230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.416660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] PATH_INFO: `/v3/projects/28d82c7df412404fac16026ea268db02` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.418650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d82c7df412404fac16026ea268db02)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:51.421035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:51.424625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:51.455517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f35b9b73fe49c6a6f4168507dae294', 'request_id': 'req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.464888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f35b9b73fe49c6a6f4168507dae294', 'request_id': 'req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.474007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f35b9b73fe49c6a6f4168507dae294', 'request_id': 'req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.499175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] The token cache is being invalidate because project 16f35b9b73fe49c6a6f4168507dae294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:51.499175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.503501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ea18f1a0-3ce8-4730-886f-b6ac4490b0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.505568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 918/1835] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:51 2022] DELETE /identity/v3/projects/16f35b9b73fe49c6a6f4168507dae294 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:51.510896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea43ff-5d78-4a4d-a5a7-12dff972ea48 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.511259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea43ff-5d78-4a4d-a5a7-12dff972ea48 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.511532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea43ff-5d78-4a4d-a5a7-12dff972ea48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.537790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-14ea43ff-5d78-4a4d-a5a7-12dff972ea48 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:51.539076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-14ea43ff-5d78-4a4d-a5a7-12dff972ea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:51.540510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d82c7df412404fac16026ea268db02', 'request_id': 'req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.553416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d82c7df412404fac16026ea268db02', 'request_id': 'req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.570358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d82c7df412404fac16026ea268db02', 'request_id': 'req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.593516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] The token cache is being invalidate because project 28d82c7df412404fac16026ea268db02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:51.594175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.595369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.598140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 918/1836] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:51 2022] DELETE /identity/v3/projects/28d82c7df412404fac16026ea268db02 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:51.602595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:51.608853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:51.664399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 919/1837] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:51.672259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.672768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.673144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.685651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:51.707351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4aac3-dd4d-49d5-a438-fa85fa1705a4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:51.720182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:51.721107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.722532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.724681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:51.725070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] PATH_INFO: `/v3/users/38db97fabfbb45c28066e377a1319753` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.726105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:51.726943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38db97fabfbb45c28066e377a1319753)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:51.733302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:51.762782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 920/1838] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:51.767900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:51.777386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcff7bc-8b6c-41e1-8d25-9bdd0898f126 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:51.796998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:51.796998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.800150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.800428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.801133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:51.801434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:51.805037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:51.833029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38db97fabfbb45c28066e377a1319753'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.834472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38db97fabfbb45c28066e377a1319753'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.846584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 921/1839] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:06:51.851302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:51.858033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38db97fabfbb45c28066e377a1319753'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.862212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5596f-2115-42b3-bab7-f260924fdaf7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:51.875398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38db97fabfbb45c28066e377a1319753'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.876939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38db97fabfbb45c28066e377a1319753'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:51.877892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:51.878536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:51.878813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:51.879117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:51.879764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:51.881699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:51.930193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 919/1840] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:06:51 2022] DELETE /identity/v3/users/38db97fabfbb45c28066e377a1319753 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:51.990177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 922/1841] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:51 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:51.994707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.001519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:52.208852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2ec02-26dd-49b2-b38f-0c6051b5efb2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:52.306718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:52.307506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.307843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.308122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.308979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:52.311225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:52.338586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 920/1842] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:51 2022] POST /identity/v3/users => generated 291 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:52.345819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.361970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41bf7a-089b-4a1a-9c7e-f32b3e48f73c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:52.369307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:52.370034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.370330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.370657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.371336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:52.373089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:52.394653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 923/1843] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:52.398919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.416832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e49ac0-b1d3-4878-8ff1-e075b44d5dd6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:52.424495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:52.425417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.426040 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.426992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.428058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:52.430365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f850835-82eb-47fa-932d-5200675d139d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:52.439941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:52.459964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 921/1844] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:06:52.464489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.478149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d3c2d-1ed2-494f-a214-846d2ed717fe admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:52.485232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:52.489781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.490069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.490351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] PATH_INFO: `/v3/projects/c5f787c561a24c02a97f227fa6e8344f/users/6860f79da8874ea98b88903b7062b14a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.512121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f787c561a24c02a97f227fa6e8344f, user_id=6860f79da8874ea98b88903b7062b14a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:52.530838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:52.551192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 924/1845] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:52 2022] PUT /identity/v3/projects/c5f787c561a24c02a97f227fa6e8344f/users/6860f79da8874ea98b88903b7062b14a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:52.561464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.562140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.562417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.610167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] MFA Rules not processed for user `6860f79da8874ea98b88903b7062b14a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:52.612201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:52.772861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.775678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 922/1846] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:52.876032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:52.918855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3eec4-b111-4ca2-9e37-6fcd58686f5c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:52.929697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:52.930615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:52.931741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:52.931741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:52.932047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:52.934291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:52.960993 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:52.962808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:52.976047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 925/1847] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:52.981515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.001407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4732e85-1ed3-4865-91d5-54fed6b1cc36 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.101510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beece641-33d7-4453-92a9-2483e61ea73e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.108287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.109085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.109462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.109665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.110298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.115742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.193837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 926/1848] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:52 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:53.203262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.249932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2df02a-faf4-484b-94cd-ed72af847e16 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.264091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.264740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.265064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.265343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.276904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.278797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.282132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.282589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] PATH_INFO: `/v3/projects/a1fb712ac4714f24bf16a5566acbae6e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.284221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fb712ac4714f24bf16a5566acbae6e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.286978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.293518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.296833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.335738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 927/1849] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:53.337112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.407081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068596-9427-4bdb-b64f-d8efa9e4bda5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.464856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fb712ac4714f24bf16a5566acbae6e', 'request_id': 'req-64cba5bf-292e-4146-88a1-d35abd4c4fe2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:53.494103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fb712ac4714f24bf16a5566acbae6e', 'request_id': 'req-64cba5bf-292e-4146-88a1-d35abd4c4fe2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:53.519002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fb712ac4714f24bf16a5566acbae6e', 'request_id': 'req-64cba5bf-292e-4146-88a1-d35abd4c4fe2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:53.558524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] The token cache is being invalidate because project a1fb712ac4714f24bf16a5566acbae6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:06:53.564150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:53.565627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64cba5bf-292e-4146-88a1-d35abd4c4fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:06:53.570061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 923/1850] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:06:52 2022] DELETE /identity/v3/projects/a1fb712ac4714f24bf16a5566acbae6e => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:53.574885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cba86-e27f-4d0e-ac6b-f9db421623de None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.575201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cba86-e27f-4d0e-ac6b-f9db421623de None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.575466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cba86-e27f-4d0e-ac6b-f9db421623de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.645691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-966cba86-e27f-4d0e-ac6b-f9db421623de None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:53.648305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-966cba86-e27f-4d0e-ac6b-f9db421623de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:53.649782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.650844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.651200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.651468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] PATH_INFO: `/v3/projects/5b2488cbcb474806a5a8401dfc82e65b/users/5250e954a5de4e96bd2e07e15f7ce21d/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.672636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 924/1851] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:53.686978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.687540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.688065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.706078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2488cbcb474806a5a8401dfc82e65b, user_id=5250e954a5de4e96bd2e07e15f7ce21d, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.709441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.741543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:53.743398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:53.750279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 928/1852] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:53 2022] PUT /identity/v3/projects/5b2488cbcb474806a5a8401dfc82e65b/users/5250e954a5de4e96bd2e07e15f7ce21d/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:53.759494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:53.761214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.768809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:53.865209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:53.915421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 925/1853] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:53.918322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.938898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783c70-1e6e-4212-bb28-749c58632c23 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.944633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.945436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.946162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.947074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.947735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.949323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.967230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 929/1854] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:53.967923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f1d34-9950-480c-97ef-895523e2f004 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.969274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:53.982037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d526-abfb-44e8-9a0c-d294af8fd502 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:53.982681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.983352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.983615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.984061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.985278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:06:53.985278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:53.989691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:53.990587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:53.991180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:53.991688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:53.992476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] PATH_INFO: `/v3/projects/5b2488cbcb474806a5a8401dfc82e65b/users/5250e954a5de4e96bd2e07e15f7ce21d/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:53.996283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2488cbcb474806a5a8401dfc82e65b, user_id=5250e954a5de4e96bd2e07e15f7ce21d, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.007362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb0c82-a509-4d16-9084-0e6809c85e6e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.011845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 926/1855] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:06:54.015654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.025719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 930/1856] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:53 2022] PUT /identity/v3/projects/5b2488cbcb474806a5a8401dfc82e65b/users/5250e954a5de4e96bd2e07e15f7ce21d/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:54.028812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa095b86-20c4-43ec-86d8-3b6c5db4ae8a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.036792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.037424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.037864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.097407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] MFA Rules not processed for user `5250e954a5de4e96bd2e07e15f7ce21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:54.099145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.128341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.129862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.130149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.130467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.131173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.133496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.176591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 927/1857] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:54.183373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.248848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 931/1858] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:06:54.266838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.273176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.348731 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4e70-f946-45c2-820d-4fce10cfb785 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.355301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.356203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.356672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.357096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.357933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.360131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b4160-99ca-4a9f-8025-ab357bff12ae admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.377052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065f68-86ad-460d-a761-c49fd4093067 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.402958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 928/1859] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:54.404275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e6e7-f312-4e3a-8965-25ca186e928f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.404837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e6e7-f312-4e3a-8965-25ca186e928f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.405181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8e6e7-f312-4e3a-8965-25ca186e928f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.406471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 929/1860] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:06:54.408470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8e6e7-f312-4e3a-8965-25ca186e928f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.433948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8e6e7-f312-4e3a-8965-25ca186e928f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.441210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.441826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.442098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.442822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.443610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.445465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecfbd7-4c36-4023-b600-ac50a591988d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.459789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.460486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 930/1861] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:54.461204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.461509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.461826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.461936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.462163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.462548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.471686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:54.475723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:54.487829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:54.489051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.538217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.539766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.557634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 932/1862] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:06:54.558501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.581218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 931/1863] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:54.586609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f19a-625e-4d07-bf5d-a43bb416b797 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.591751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.592329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.592616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.592888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.594163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.595854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.598938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.601661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.607632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 933/1864] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:54.611448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.617785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f59b2a-1712-416a-9fca-475b8eb0abbf None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.627748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.628856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.629223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.629406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.632077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073897d3-4285-4029-9a52-d60d9b437b38 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.641415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.643316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.643713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.644190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] PATH_INFO: `/v3/projects/a38164e09c514449a28a693bfda0678c/users/fff4a4631357411f9407c2ef2b2801d4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.655441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.657130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.665501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38164e09c514449a28a693bfda0678c, user_id=fff4a4631357411f9407c2ef2b2801d4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.667660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e241797-a4a8-4d21-ba67-013ce59bba27 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.683256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 932/1865] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:54.685041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.696464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2647a51-9bd6-42a0-b8ec-00667b3d28c3 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.703555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 934/1866] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:54 2022] PUT /identity/v3/projects/a38164e09c514449a28a693bfda0678c/users/fff4a4631357411f9407c2ef2b2801d4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:06:54.705367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.705754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.706128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.746015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] MFA Rules not processed for user `fff4a4631357411f9407c2ef2b2801d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:54.747465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.768162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.768162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.768162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.768550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.770158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:54.844275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 935/1867] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:54.848964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:54.851552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.853974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.875616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 933/1868] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:06:54.937370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e626c-8dd9-4d51-9139-27701d976690 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:54.944078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:54.945111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:54.945669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:54.945937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:54.946833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:54.948830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:54.956579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:55.004831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 936/1869] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:06:54 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:06:55.010265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.025438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a7b3a-907a-4ef1-8528-043563fcc5e8 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.068432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:06:55.107965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.109735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.109996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.110273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.110955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.113611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.137843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 934/1870] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:06:55 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:06:55.141973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.157498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab0c1e-fa0f-4ba7-8f98-a28ccbc2583b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.163405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.164267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.164676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.164835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.165467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.167166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.181075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 937/1871] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:55.185527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.211841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d969fa-112e-4c90-a002-f2f09754e1b7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.231545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.232777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.233068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.233348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] PATH_INFO: `/v3/projects/ee24f10922f94f34b9321b802c579983/users/85637432b0474cb699641db8376eeee4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.266788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee24f10922f94f34b9321b802c579983, user_id=85637432b0474cb699641db8376eeee4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.269599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.298081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 935/1872] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:55 2022] PUT /identity/v3/projects/ee24f10922f94f34b9321b802c579983/users/85637432b0474cb699641db8376eeee4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:55.302385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.313085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8124af-0805-49de-a841-2bcd3266f34a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.382152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.382768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.383074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.383347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.383989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.385579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defebb31-97d5-4275-96e8-592da70d54b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.399061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 938/1873] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:06:55.408525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.421331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcaa75-559b-4ac2-bf48-7543f9f49307 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.427775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.428431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.428712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.428983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] PATH_INFO: `/v3/projects/ee24f10922f94f34b9321b802c579983/users/85637432b0474cb699641db8376eeee4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.430926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee24f10922f94f34b9321b802c579983, user_id=85637432b0474cb699641db8376eeee4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.433724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.446797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 936/1874] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:06:55 2022] PUT /identity/v3/projects/ee24f10922f94f34b9321b802c579983/users/85637432b0474cb699641db8376eeee4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:06:55.457576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.458623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.459005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.493085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] MFA Rules not processed for user `85637432b0474cb699641db8376eeee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:06:55.495483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:06:55.569493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 939/1875] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:06:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:06:55.580398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:06:55.688686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6206c8-4aba-4938-941a-17258e84e38e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:06:55.693124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:06:55.694585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:06:55.694997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:06:55.695373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:06:55.783015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:06:55.783015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c31af1-b995-4dc8-93ba-7bcbf03177c7 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:06:55.791217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 937/1876] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:06:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:00.016104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:00.084380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:00.089107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73f573-cdf6-4629-8d7a-862737176081 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:00.096107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:00.096756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:00.097753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:00.098265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] PATH_INFO: `/v3/users/6860f79da8874ea98b88903b7062b14a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:00.099627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6860f79da8874ea98b88903b7062b14a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:00.101667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:00.182185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6860f79da8874ea98b88903b7062b14a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:00.196221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6860f79da8874ea98b88903b7062b14a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:00.214518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6860f79da8874ea98b88903b7062b14a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:00.217146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6860f79da8874ea98b88903b7062b14a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:00.218154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6860f79da8874ea98b88903b7062b14a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:00.249736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 940/1877] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:00 2022] DELETE /identity/v3/users/6860f79da8874ea98b88903b7062b14a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:00.251627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:00.252196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:00.252285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:00.305676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:00.307821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:00.386046 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:00.420306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 938/1878] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:00.444830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:00.458598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:00.556582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414a114-4064-4333-8fd8-edc97f7d6145 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:00.560558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:00.561178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:00.561452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:00.561825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:00.574891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:00.576563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:00.599081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 941/1879] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:00.889839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:00.896136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:00.914985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f00aa-6942-4463-8a42-64f5153ac580 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:00.927074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:00.927641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:00.927927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:00.928189 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] PATH_INFO: `/v3/projects/c5f787c561a24c02a97f227fa6e8344f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:00.929501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f787c561a24c02a97f227fa6e8344f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:00.932874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:01.014974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f787c561a24c02a97f227fa6e8344f', 'request_id': 'req-67bd354c-a0cf-4738-b8f9-8d9b2778364a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.022805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f787c561a24c02a97f227fa6e8344f', 'request_id': 'req-67bd354c-a0cf-4738-b8f9-8d9b2778364a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.033179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f787c561a24c02a97f227fa6e8344f', 'request_id': 'req-67bd354c-a0cf-4738-b8f9-8d9b2778364a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.052288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] The token cache is being invalidate because project c5f787c561a24c02a97f227fa6e8344f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:01.052616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.053332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.058818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 939/1880] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:00 2022] DELETE /identity/v3/projects/c5f787c561a24c02a97f227fa6e8344f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:01.059827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:01.072392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b69a-65e2-447b-9939-ee71d84feb28 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:01.148594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:01.149525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:01.149967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:01.150548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] PATH_INFO: `/v3/users/5250e954a5de4e96bd2e07e15f7ce21d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:01.152634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5250e954a5de4e96bd2e07e15f7ce21d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:01.157436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:01.193046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:01.226114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5250e954a5de4e96bd2e07e15f7ce21d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.241866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5250e954a5de4e96bd2e07e15f7ce21d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.248773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5250e954a5de4e96bd2e07e15f7ce21d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.249754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5250e954a5de4e96bd2e07e15f7ce21d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.250511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6ffc3f50-cef2-4f27-ba1f-1dd74111538e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5250e954a5de4e96bd2e07e15f7ce21d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.266794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 942/1881] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:01 2022] DELETE /identity/v3/users/5250e954a5de4e96bd2e07e15f7ce21d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:01.748323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:01.750376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:01.769960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd354c-a0cf-4738-b8f9-8d9b2778364a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:01.852507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:01.853147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:01.853430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:01.853698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] PATH_INFO: `/v3/projects/5b2488cbcb474806a5a8401dfc82e65b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:01.855352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2488cbcb474806a5a8401dfc82e65b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:01.857552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:01.928956 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2488cbcb474806a5a8401dfc82e65b', 'request_id': 'req-0165e7f5-8b26-4da5-9017-3a4d343425c3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.936679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2488cbcb474806a5a8401dfc82e65b', 'request_id': 'req-0165e7f5-8b26-4da5-9017-3a4d343425c3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.943999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2488cbcb474806a5a8401dfc82e65b', 'request_id': 'req-0165e7f5-8b26-4da5-9017-3a4d343425c3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.948088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:01.966753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] The token cache is being invalidate because project 5b2488cbcb474806a5a8401dfc82e65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:01.967267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.968967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0165e7f5-8b26-4da5-9017-3a4d343425c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:01.970634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 940/1882] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:01 2022] DELETE /identity/v3/projects/5b2488cbcb474806a5a8401dfc82e65b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:01.977494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:01.977494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:01.977494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.005710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:02.007539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:02.077674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 943/1883] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:02.085468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.085753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.086059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.107415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:02.108764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:02.118013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 941/1884] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:02.122349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.133272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0d0b-87d1-4266-a105-1b9af3556c09 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.139060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.139817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.140136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.140383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.141041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:02.141335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.142908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.156125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 944/1885] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:02.160698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.169931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b402317-4ae7-4dc1-be21-d297a954e510 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.194035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.194662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.194941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.195255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.195903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.197874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.233946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 942/1886] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:02.237694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.249837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5844cd8-df7a-41e1-885b-17ae6ca51a23 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.330498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.331250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.331556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.331838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.333480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.335515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.358430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 945/1887] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:02.362968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.374726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cb8fe-141e-48a6-ae88-6992c3829500 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.379667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.380274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.380698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.380967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.381650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.383332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.394718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 943/1888] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:02.399166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.410352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045ce2-36f4-4eca-80c1-b8432768cdcb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.415101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.415707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.415983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.416252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.416889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.418753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b888f55-54fe-4e05-ac8b-0b2a5f133ad6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.439630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 946/1889] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:02.440571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.460569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eadeb0-32b7-4cf9-acf1-052ec2f192f3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.463681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.463681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.463681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.463681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] PATH_INFO: `/v3/projects/0a08ab3142884f67a25899e3d8d378dd/users/e1ceab6514604feb969135d72d88a332/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.482899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a08ab3142884f67a25899e3d8d378dd, user_id=e1ceab6514604feb969135d72d88a332, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.486298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.516573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 944/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:02 2022] PUT /identity/v3/projects/0a08ab3142884f67a25899e3d8d378dd/users/e1ceab6514604feb969135d72d88a332/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:02.519899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.520219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.520518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.530133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:02.564622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] MFA Rules not processed for user `e1ceab6514604feb969135d72d88a332`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:02.566273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:02.648758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 947/1891] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:02.653094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.774283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534341a-bc81-4fdb-9ebe-758e650894eb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.781815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.782538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.783683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.783970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.784690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.786923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.850051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 945/1892] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:02.859613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:02.876409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435138b4-7274-44a4-9fde-024d23330eeb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:02.954034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:02.954685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:02.954980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:02.955432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:02.956114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:02.958219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:02.986595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 948/1893] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:02 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:02.990828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.001781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad76c-946a-48d6-8ae8-53a18101f269 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.006383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.007060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.007363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.007819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.008494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:03.010186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:03.025270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 946/1894] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:03.027521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.040424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59c3e3-6c86-45fb-ba82-80074682fe4e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.045871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.046603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.046960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.047313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] PATH_INFO: `/v3/projects/f57d452270874546b6f7becb0eebab7b/users/37399cea0f0d43ecbe59a4956e8f262e/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.066248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57d452270874546b6f7becb0eebab7b, user_id=37399cea0f0d43ecbe59a4956e8f262e, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:03.070329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:03.105410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 949/1895] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:03 2022] PUT /identity/v3/projects/f57d452270874546b6f7becb0eebab7b/users/37399cea0f0d43ecbe59a4956e8f262e/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:03.107440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.121627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99400e-ce71-4a0f-bc30-9984467c5d3b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.234425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.235774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.236106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.236434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.237100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:03.239467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d4f3-4a3e-42ec-a1f1-b6c3e82e75ff admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:03.255202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 947/1896] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:03.257960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.271512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74513f-0757-4285-a675-20e0cfd79302 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.288134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.289834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.290285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.290714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] PATH_INFO: `/v3/projects/f57d452270874546b6f7becb0eebab7b/users/37399cea0f0d43ecbe59a4956e8f262e/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.293168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57d452270874546b6f7becb0eebab7b, user_id=37399cea0f0d43ecbe59a4956e8f262e, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:03.296256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:03.315018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 950/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:03 2022] PUT /identity/v3/projects/f57d452270874546b6f7becb0eebab7b/users/37399cea0f0d43ecbe59a4956e8f262e/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:03.327007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.327682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.328129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.340409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:03.366504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] MFA Rules not processed for user `37399cea0f0d43ecbe59a4956e8f262e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:03.368030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:03.439516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 948/1898] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:03.450422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.452401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:03.513994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:03.519560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb1920-925f-40b5-b9d0-0251b03070fd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.524369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.525012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.525295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.525602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.535183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:03.536602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:03.546441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 951/1899] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:03.895660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:03.898270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:03.909606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d1755-7c3c-470a-987b-412800b97cc3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:03.913836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:03.914526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:03.914874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:03.915241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:03.917177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:04.007845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:04.009454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d29fe-7a4b-4ec2-9f1a-2139361748a8 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:04.021789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 949/1900] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:04.032197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:04.129558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914a8b-bd0a-4db9-92ca-b6e76b0c9b56 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:04.138306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:04.143144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:04.143144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:04.143144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] PATH_INFO: `/v3/users/e1ceab6514604feb969135d72d88a332` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:04.144896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ceab6514604feb969135d72d88a332)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:04.150797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:04.238567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ceab6514604feb969135d72d88a332'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:04.290934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ceab6514604feb969135d72d88a332'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:04.298766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ceab6514604feb969135d72d88a332'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:04.299626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ceab6514604feb969135d72d88a332'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:04.300298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ceab6514604feb969135d72d88a332'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:04.338796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 952/1901] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:04 2022] DELETE /identity/v3/users/e1ceab6514604feb969135d72d88a332 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:04.343878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214d89-12ac-4b7d-9a0a-02bfe9b847c7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:04.344276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214d89-12ac-4b7d-9a0a-02bfe9b847c7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:04.344782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214d89-12ac-4b7d-9a0a-02bfe9b847c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:04.391584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bb214d89-12ac-4b7d-9a0a-02bfe9b847c7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:04.397539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bb214d89-12ac-4b7d-9a0a-02bfe9b847c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:04.591256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 950/1902] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:04.592285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:04.594108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:04.735764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab733d0a-1787-4e53-a699-618e4425e422 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:04.745548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:04.747771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:04.749514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:04.749798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:04.776793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:04.778258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e7279-30ad-4141-86fc-57654291c65e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:04.800057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 953/1903] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:04.920308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:04.924065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:04.924747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:04.926304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 951/1904] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:07:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:07:04.945189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:04.945576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:04.945995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:04.999843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:05.001381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:05.009465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:05.143222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 954/1905] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:07:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:05.296457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:05.298252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:05.317414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab4e41-427d-48cd-b522-c1fa0e01cae3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:05.324229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:05.324999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:05.325274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:05.325571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:05.335094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:05.336813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:05.348508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 952/1906] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:05.607701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:05.609272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:05.622892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a076af5-d329-4600-89ce-1e89906c07cb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:05.627592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:05.628320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:05.628524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:05.628789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] PATH_INFO: `/v3/projects/0a08ab3142884f67a25899e3d8d378dd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:05.629914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a08ab3142884f67a25899e3d8d378dd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:05.631939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:05.715148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a08ab3142884f67a25899e3d8d378dd', 'request_id': 'req-936cdb3f-a412-439b-80ea-3ebb667ec9a7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.722267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a08ab3142884f67a25899e3d8d378dd', 'request_id': 'req-936cdb3f-a412-439b-80ea-3ebb667ec9a7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.727988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a08ab3142884f67a25899e3d8d378dd', 'request_id': 'req-936cdb3f-a412-439b-80ea-3ebb667ec9a7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.741750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] The token cache is being invalidate because project 0a08ab3142884f67a25899e3d8d378dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:05.742266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.742901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.744517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 955/1907] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:05 2022] DELETE /identity/v3/projects/0a08ab3142884f67a25899e3d8d378dd => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:05.748824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:05.759669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96117d6-b440-491b-b20b-22443e32a619 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:05.764651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:05.822896 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:05.823621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:05.823969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:05.824239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] PATH_INFO: `/v3/users/37399cea0f0d43ecbe59a4956e8f262e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:05.825484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37399cea0f0d43ecbe59a4956e8f262e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:05.827541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:05.879523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37399cea0f0d43ecbe59a4956e8f262e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.880173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37399cea0f0d43ecbe59a4956e8f262e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.886940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37399cea0f0d43ecbe59a4956e8f262e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.897175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37399cea0f0d43ecbe59a4956e8f262e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.897819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3ec2a85f-09f9-4f2e-98ea-29b04b0302c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37399cea0f0d43ecbe59a4956e8f262e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:05.917342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 953/1908] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:05 2022] DELETE /identity/v3/users/37399cea0f0d43ecbe59a4956e8f262e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:06.236988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.238690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:06.255846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cdb3f-a412-439b-80ea-3ebb667ec9a7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.318381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40540537-d180-4687-b18a-0d416044540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.319238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540537-d180-4687-b18a-0d416044540c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.319521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540537-d180-4687-b18a-0d416044540c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.319949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540537-d180-4687-b18a-0d416044540c admin admin] PATH_INFO: `/v3/projects/f57d452270874546b6f7becb0eebab7b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.321353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40540537-d180-4687-b18a-0d416044540c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57d452270874546b6f7becb0eebab7b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.323327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40540537-d180-4687-b18a-0d416044540c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.356193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:06.375717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57d452270874546b6f7becb0eebab7b', 'request_id': 'req-40540537-d180-4687-b18a-0d416044540c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:06.382297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57d452270874546b6f7becb0eebab7b', 'request_id': 'req-40540537-d180-4687-b18a-0d416044540c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:06.388864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57d452270874546b6f7becb0eebab7b', 'request_id': 'req-40540537-d180-4687-b18a-0d416044540c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:06.402357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] The token cache is being invalidate because project f57d452270874546b6f7becb0eebab7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:06.402643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:06.403316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-40540537-d180-4687-b18a-0d416044540c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:06.404818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 956/1909] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:06 2022] DELETE /identity/v3/projects/f57d452270874546b6f7becb0eebab7b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:06.410893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2d7d-3318-413f-a349-55ded266803c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.411411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2d7d-3318-413f-a349-55ded266803c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.411633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a2d7d-3318-413f-a349-55ded266803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.432440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-569a2d7d-3318-413f-a349-55ded266803c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:06.433662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-569a2d7d-3318-413f-a349-55ded266803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:06.495975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 954/1910] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:06.504075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.504363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.504634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.532526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:06.533741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:06.542934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 957/1911] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:06.547292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2d7d-3318-413f-a349-55ded266803c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.557424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a2d7d-3318-413f-a349-55ded266803c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.562135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.562733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.563006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.563306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.563954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:06.564217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.565811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.579124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 955/1912] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:06.584221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.594032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b26193-3305-4448-bc0f-de2c7e63a25d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.598739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.599398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.599674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.599963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.600649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.602626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.638446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 958/1913] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:06 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:06.639981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.653256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126bde7-eae3-4a27-9b93-cb5c93d24428 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.669079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:06.722516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.723162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.723446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.723732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.724351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.726267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.749717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 956/1914] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:06 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:06.754957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.766367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bdc36-de0b-4533-b0a3-b63a14d3c711 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.771164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.771783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.772078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.772330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.772966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.774556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.785664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 959/1915] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:06.789999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.801575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec7939-3677-4980-824c-5460657fa395 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.807109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.807800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.808025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.808297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.808944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.810552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b502-9579-46f2-a739-14d66c0f63d7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.824305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 957/1916] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:06.829187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:06.840547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073438a-3c6d-4eb4-abde-326d6f7c7afe admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:06.846165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:06.846827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.847142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.847418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] PATH_INFO: `/v3/projects/82dd303dbcd34a0198cf939cc6feda9e/users/9ee82d34660344a4a68b23cab100046e/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.869745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82dd303dbcd34a0198cf939cc6feda9e, user_id=9ee82d34660344a4a68b23cab100046e, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:06.872695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:06.888901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 960/1917] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:06 2022] PUT /identity/v3/projects/82dd303dbcd34a0198cf939cc6feda9e/users/9ee82d34660344a4a68b23cab100046e/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:06.901580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:06.901955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:06.902350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:06.938677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] MFA Rules not processed for user `9ee82d34660344a4a68b23cab100046e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:06.940038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:07.005627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 958/1918] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:07.018687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:07.020531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:07.082039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c3934-ddd8-4266-abe3-7595283d0de0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:07.086718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:07.087363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:07.087643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:07.087909 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:07.097110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:07.098729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a761e9a-d0a2-40ba-a746-eecae2c8c5df service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:07.109420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 961/1919] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:12.522246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:12.523875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:12.605954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254be39-e665-4236-b4d7-1af9ba1ab4f0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:12.611000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:12.611701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:12.611988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:12.612259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] PATH_INFO: `/v3/users/cd25f67637b94fd9a43f40a8f7b2d6b9` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:12.613394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd25f67637b94fd9a43f40a8f7b2d6b9)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:12.615756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:12.649686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:12.683611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd25f67637b94fd9a43f40a8f7b2d6b9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:12.684350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd25f67637b94fd9a43f40a8f7b2d6b9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:12.692000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd25f67637b94fd9a43f40a8f7b2d6b9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:12.700109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd25f67637b94fd9a43f40a8f7b2d6b9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:12.700740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd25f67637b94fd9a43f40a8f7b2d6b9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:12.723424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 959/1920] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:12 2022] DELETE /identity/v3/users/cd25f67637b94fd9a43f40a8f7b2d6b9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:12.725415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:12.725901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:12.726006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:12.745532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:12.746733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:12.804273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:12.819457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 962/1921] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:12.830030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:12.831591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:12.909673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dda183-2c64-458c-9581-e79b73f47844 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:12.913336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:12.914084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:12.914364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:12.914636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:12.924679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:12.926182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:12.937454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 960/1922] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:13.380360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:13.382399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:13.400721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da09ba2-b8d7-4895-a700-603641f83756 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:13.407181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:13.407832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:13.408119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:13.408390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] PATH_INFO: `/v3/projects/a96083b38cc744588699cc530ba7a917` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:13.409657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96083b38cc744588699cc530ba7a917)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:13.411535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:13.497794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96083b38cc744588699cc530ba7a917', 'request_id': 'req-559a0048-c8d6-47a6-981d-28d9addbc202'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.511252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96083b38cc744588699cc530ba7a917', 'request_id': 'req-559a0048-c8d6-47a6-981d-28d9addbc202'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.519122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96083b38cc744588699cc530ba7a917', 'request_id': 'req-559a0048-c8d6-47a6-981d-28d9addbc202'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.535973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] The token cache is being invalidate because project a96083b38cc744588699cc530ba7a917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:13.536335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.537052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.538978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 963/1923] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:13 2022] DELETE /identity/v3/projects/a96083b38cc744588699cc530ba7a917 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:13.542994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:13.555527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef7bad-dd83-4b19-9f28-9533f4c80f8a service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:13.633002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:13.633706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:13.633963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:13.634229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] PATH_INFO: `/v3/users/8e039742883c42849d617d1707e9932f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:13.635401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e039742883c42849d617d1707e9932f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:13.637328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:13.689258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e039742883c42849d617d1707e9932f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.690752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e039742883c42849d617d1707e9932f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.692117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:13.699432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e039742883c42849d617d1707e9932f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.710967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e039742883c42849d617d1707e9932f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.711675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-77878622-8599-4a9b-8ac3-cecab9d8e4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e039742883c42849d617d1707e9932f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:13.729871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 961/1924] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:13 2022] DELETE /identity/v3/users/8e039742883c42849d617d1707e9932f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:14.139772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.141288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.159174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a0048-c8d6-47a6-981d-28d9addbc202 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.272958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.273708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.274012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.274286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] PATH_INFO: `/v3/projects/1a029e73ff834f7e818444ef2d4ffaac` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.275802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a029e73ff834f7e818444ef2d4ffaac)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.277873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.372032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a029e73ff834f7e818444ef2d4ffaac', 'request_id': 'req-b4452e97-0ee0-4beb-8253-e93d213eedbe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.379695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a029e73ff834f7e818444ef2d4ffaac', 'request_id': 'req-b4452e97-0ee0-4beb-8253-e93d213eedbe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.391101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a029e73ff834f7e818444ef2d4ffaac', 'request_id': 'req-b4452e97-0ee0-4beb-8253-e93d213eedbe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.415216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] The token cache is being invalidate because project 1a029e73ff834f7e818444ef2d4ffaac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:14.415764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.417079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4452e97-0ee0-4beb-8253-e93d213eedbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.419218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 964/1925] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:14 2022] DELETE /identity/v3/projects/1a029e73ff834f7e818444ef2d4ffaac => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:14.424407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.424695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.424973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.446832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:14.448101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.529340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 962/1926] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:14.538400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.538686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.538953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.560943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:14.562097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.570859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 965/1927] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:14.575296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.586390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f2941-498f-4dfd-94c3-b806af3b1d69 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.591166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.591768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.592051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.592380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.593346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:14.593654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.599353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.612326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 963/1928] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:07:14.617440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.622462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:14.631658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f89c47-84a9-419a-82bf-2527087cf852 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.643978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.645072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.645486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.645883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.646695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.648957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.666277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.669372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.684039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70c43d-534d-4715-a28b-2dca8a11aa91 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.684268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 966/1929] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:14.688397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.754418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2305-f655-44de-9181-99ae64badae5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.755476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.756115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.756394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.756661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] PATH_INFO: `/v3/users/fff4a4631357411f9407c2ef2b2801d4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.757819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff4a4631357411f9407c2ef2b2801d4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.759782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.760501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.761118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.761591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.761869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.762862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.765059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.789493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 967/1930] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:14.793776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.805207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78848e-bfad-4872-97ff-6fdecc799de3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.810489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.811225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.811664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.812055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.812753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.814378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.825850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 968/1931] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:14.830738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.837764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff4a4631357411f9407c2ef2b2801d4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.838667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff4a4631357411f9407c2ef2b2801d4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.841527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601ea30-91fe-4ef9-bd6f-74e1214567d4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.845945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:14.846310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.847062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.847325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.847555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.848155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.848639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff4a4631357411f9407c2ef2b2801d4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.849761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.858715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff4a4631357411f9407c2ef2b2801d4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.860254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ec27d28-d4ff-49b8-b7d9-769bc1430068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff4a4631357411f9407c2ef2b2801d4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:14.865653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 969/1932] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:14.866764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:14.868578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.881467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ce73d-7402-4bab-b0fd-2ac4086e9018 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:14.886612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:14.887294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.887625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.887908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] PATH_INFO: `/v3/projects/d6a6a62c65264c0182caea311a8c204a/users/41ee81d0468a487687f1710b02155237/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.890111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 964/1933] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:14 2022] DELETE /identity/v3/users/fff4a4631357411f9407c2ef2b2801d4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:14.891551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.891825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.892112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.904472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a6a62c65264c0182caea311a8c204a, user_id=41ee81d0468a487687f1710b02155237, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:14.907993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da34f7-518a-49aa-9bb6-1ca129650c40 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:14.915326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:14.916730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:14.928094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 970/1934] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:14 2022] PUT /identity/v3/projects/d6a6a62c65264c0182caea311a8c204a/users/41ee81d0468a487687f1710b02155237/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:14.938102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:14.938625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:14.939204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:14.969702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] MFA Rules not processed for user `41ee81d0468a487687f1710b02155237`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:14.971115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:15.016665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 965/1935] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:15.027417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.029005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:15.043465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 971/1936] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:15.046549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.063885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:15.101905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3a06d-57d9-4561-ae49-2c09a995d1b9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.126466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74ad3-5913-4e1b-a43a-4ba3190d0566 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.132433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.133123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.133542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.133939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.135196 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.137744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.162376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.163341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.163815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.164885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.174160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 972/1937] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:15 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:15.176190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.178164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.178587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.202736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 966/1938] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:15.297801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05741789-aeac-426e-9c85-eacbe7c49f37 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.303366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.304305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.304644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.305007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.306215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.308351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.345969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 973/1939] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:15 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:15.350052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.363000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb101a-bb79-4765-aa04-80794a4075cb service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.377909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.378588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.378977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.379509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.380306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.382722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.412730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 967/1940] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:15.419238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.453563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92c13-baec-40d9-97fb-3ee87b13b4a2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.468763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.470168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.470574 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.470950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] PATH_INFO: `/v3/projects/1614f5e576c441aba2573912771ad708/users/780c6a76e2aa4ec99ab4853454b8b942/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.493170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1614f5e576c441aba2573912771ad708, user_id=780c6a76e2aa4ec99ab4853454b8b942, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.496184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.517372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 974/1941] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:15 2022] PUT /identity/v3/projects/1614f5e576c441aba2573912771ad708/users/780c6a76e2aa4ec99ab4853454b8b942/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:15.522699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.536537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd4e8-2c60-4cfa-8784-9a772887a395 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.558163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:15.624474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.625520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.626037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.626381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.627300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.630179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.657085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.659077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 968/1942] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:15.673968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f531-71bb-479b-95d3-1c2b61b962ef admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.693327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.693327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.693327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.693327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] PATH_INFO: `/v3/projects/1614f5e576c441aba2573912771ad708/users/780c6a76e2aa4ec99ab4853454b8b942/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.699271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1614f5e576c441aba2573912771ad708, user_id=780c6a76e2aa4ec99ab4853454b8b942, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.702787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0c05d-4c25-4475-9c9d-62a23e518573 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.716028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:15.718290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:15.741210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f74e-c246-4e28-b94e-75d50d254020 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:15.753198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 975/1943] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:15 2022] PUT /identity/v3/projects/1614f5e576c441aba2573912771ad708/users/780c6a76e2aa4ec99ab4853454b8b942/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:15.763587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.764138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.764691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.830567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:15.831968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:15.832582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:15.833049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] PATH_INFO: `/v3/projects/a38164e09c514449a28a693bfda0678c` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:15.834769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38164e09c514449a28a693bfda0678c)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:15.843307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:15.855770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] MFA Rules not processed for user `780c6a76e2aa4ec99ab4853454b8b942`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:15.858688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:15.935569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38164e09c514449a28a693bfda0678c', 'request_id': 'req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:15.946106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38164e09c514449a28a693bfda0678c', 'request_id': 'req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:15.953007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38164e09c514449a28a693bfda0678c', 'request_id': 'req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:15.959608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:15.976084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] The token cache is being invalidate because project a38164e09c514449a28a693bfda0678c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:15.978276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:15.979812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:15.987616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 969/1944] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:15 2022] DELETE /identity/v3/projects/a38164e09c514449a28a693bfda0678c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:15.989492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:16.003142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:16.040600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 976/1945] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:16.077069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:16.091513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:16.094144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:16.103310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0cbc2-a33c-4616-9c39-4c32787a0e2f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:16.110333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:16.111181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:16.111462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:16.111767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] PATH_INFO: `/v3/users/85637432b0474cb699641db8376eeee4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:16.113373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=85637432b0474cb699641db8376eeee4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:16.116545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:16.180499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a8994-0d34-4808-a081-0c60eb9059d5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:16.181844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85637432b0474cb699641db8376eeee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.182514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85637432b0474cb699641db8376eeee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.190396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85637432b0474cb699641db8376eeee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.201567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85637432b0474cb699641db8376eeee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.202967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85637432b0474cb699641db8376eeee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.250086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 970/1946] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:15 2022] DELETE /identity/v3/users/85637432b0474cb699641db8376eeee4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:16.300787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:16.301448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:16.301915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:16.302053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:16.311956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:16.430327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:16.432488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:16.448087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 977/1947] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:16.627151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:16.628669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:16.747875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:16.750082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:16.780357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e7de-1821-4f5f-8760-8c0bb45cb75b service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:16.806704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee389ff-b751-45da-8bfd-a2df6acb87dc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:16.819995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:16.820646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:16.820928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:16.821203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] PATH_INFO: `/v3/users/41ee81d0468a487687f1710b02155237` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:16.824825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ee81d0468a487687f1710b02155237)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:16.826912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:16.904934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:16.912614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ee81d0468a487687f1710b02155237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.914156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ee81d0468a487687f1710b02155237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.933738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ee81d0468a487687f1710b02155237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.938279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:16.940252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:16.941006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ee81d0468a487687f1710b02155237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.941544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:16.942034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7c283f88-77ea-423a-a5a6-70559923a24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ee81d0468a487687f1710b02155237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:16.943896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] PATH_INFO: `/v3/projects/ee24f10922f94f34b9321b802c579983` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:16.945550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee24f10922f94f34b9321b802c579983)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:16.949081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:16.951521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:16.979534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 971/1948] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:16 2022] DELETE /identity/v3/users/41ee81d0468a487687f1710b02155237 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:16.985104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:16.985470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:16.985805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.010772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:17.012103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.028955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee24f10922f94f34b9321b802c579983', 'request_id': 'req-3df087a4-34f7-40ab-bb04-12332d87359c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.051161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee24f10922f94f34b9321b802c579983', 'request_id': 'req-3df087a4-34f7-40ab-bb04-12332d87359c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.051161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee24f10922f94f34b9321b802c579983', 'request_id': 'req-3df087a4-34f7-40ab-bb04-12332d87359c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.061540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] The token cache is being invalidate because project ee24f10922f94f34b9321b802c579983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:17.062058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.062824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3df087a4-34f7-40ab-bb04-12332d87359c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.067140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 978/1949] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:16 2022] DELETE /identity/v3/projects/ee24f10922f94f34b9321b802c579983 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:17.070338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95dacb-cf52-453e-a670-42567a18a9d2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.070620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95dacb-cf52-453e-a670-42567a18a9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.070892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95dacb-cf52-453e-a670-42567a18a9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.096099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2c95dacb-cf52-453e-a670-42567a18a9d2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:17.097472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c95dacb-cf52-453e-a670-42567a18a9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.106321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 972/1950] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:17.118163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.120019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.161661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 979/1951] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:17.170169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.170800 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.171238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.188542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e939a-62bd-42b3-aed8-4aeade05aedd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.192071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:17.192526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.192989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.193211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.193304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.193522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.208345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 980/1952] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:17.209397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.210893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.212361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.221257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 973/1953] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:17.221791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf56f5-6583-4ec2-9bc4-964c4aec0101 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.226830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.227476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.227760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.228032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.228750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:17.229062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.230774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.247528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 981/1954] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:07:17.248802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.259299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe6a3-e705-4acc-9288-9e9e8cfb71d1 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.270540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.271175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.271548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.271770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.272645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.274732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.307508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 974/1955] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:17 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:17.308771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.319792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70114492-189b-461c-9365-a7106de8f535 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.380870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.381656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.382104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.382438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.383596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.385825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.389306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:17.413705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 982/1956] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:17 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:17.414984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.425887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bbbcb-ffc7-46de-b010-fbdb6dc12393 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.430576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.431212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.431494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.431761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.432404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.434107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.445001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 975/1957] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:17.448877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.459626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8d010-fb0d-4459-8eaf-d8cd23646730 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.464542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.465143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.465419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.465689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.466356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.468613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.481029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 983/1958] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:17.485941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.498812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a511cbd-b2af-4ad9-8f46-9ab36f8d5cd3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.503401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.505087 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.511171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.512048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.512620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.512877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] PATH_INFO: `/v3/projects/41fc730ba093494ca268d17dda13b370/users/a45da0676d2d4546997d31583e4c28f7/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.519522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9646d-88cf-4464-9f0a-e76db61f6b63 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.525031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.525759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.526220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.526445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] PATH_INFO: `/v3/projects/d6a6a62c65264c0182caea311a8c204a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.528011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a6a62c65264c0182caea311a8c204a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.530958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.539669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=41fc730ba093494ca268d17dda13b370, user_id=a45da0676d2d4546997d31583e4c28f7, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.542907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325e390-1b2f-4970-9ab6-5a88785ab3ed admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.558888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 976/1959] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:17 2022] PUT /identity/v3/projects/41fc730ba093494ca268d17dda13b370/users/a45da0676d2d4546997d31583e4c28f7/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:17.578512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.578512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.578512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.607804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a6a62c65264c0182caea311a8c204a', 'request_id': 'req-b8a26617-6768-495b-b11a-fb248c9402f3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.615641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a6a62c65264c0182caea311a8c204a', 'request_id': 'req-b8a26617-6768-495b-b11a-fb248c9402f3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.621968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a6a62c65264c0182caea311a8c204a', 'request_id': 'req-b8a26617-6768-495b-b11a-fb248c9402f3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.622726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] MFA Rules not processed for user `a45da0676d2d4546997d31583e4c28f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:17.624290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.641270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] The token cache is being invalidate because project d6a6a62c65264c0182caea311a8c204a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:17.641680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.642697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.645588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 984/1960] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:17 2022] DELETE /identity/v3/projects/d6a6a62c65264c0182caea311a8c204a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:17.646065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:17.650485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.652321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.726216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 977/1961] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:17.739842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:17.741558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26617-6768-495b-b11a-fb248c9402f3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.741695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:17.746249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.746886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.747195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.747513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] PATH_INFO: `/v3/users/780c6a76e2aa4ec99ab4853454b8b942` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.748868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] RBAC: Authorizing `identity:delete_user(user_id=780c6a76e2aa4ec99ab4853454b8b942)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.750830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.807312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780c6a76e2aa4ec99ab4853454b8b942'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.812008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b504124-737c-4cee-a1f6-571e67ecbefe None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:17.817679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:INSERT=4,keystone:DELETE=15 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:17.817972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:17.818729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:17.819452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:17.819452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:17.824206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780c6a76e2aa4ec99ab4853454b8b942'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.830931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:17.832417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:17.833685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780c6a76e2aa4ec99ab4853454b8b942'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.834536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780c6a76e2aa4ec99ab4853454b8b942'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.835330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3514a915-09ce-4647-ad86-0fb60e60401f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780c6a76e2aa4ec99ab4853454b8b942'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:17.843556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 978/1962] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:17.850130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 985/1963] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:17 2022] DELETE /identity/v3/users/780c6a76e2aa4ec99ab4853454b8b942 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:18.249160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.262385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711abeb-7436-42d0-aafc-7326505ddc81 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.352030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.352697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.352971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.353236 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] PATH_INFO: `/v3/projects/1614f5e576c441aba2573912771ad708` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.354432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1614f5e576c441aba2573912771ad708)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.356379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.404240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1614f5e576c441aba2573912771ad708', 'request_id': 'req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:18.410533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1614f5e576c441aba2573912771ad708', 'request_id': 'req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:18.416168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1614f5e576c441aba2573912771ad708', 'request_id': 'req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:18.418945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:18.430264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] The token cache is being invalidate because project 1614f5e576c441aba2573912771ad708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:18.430742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:18.431383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-31cccdb7-c610-4d0c-a45f-039b0e6c1e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:18.432907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 979/1964] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:18 2022] DELETE /identity/v3/projects/1614f5e576c441aba2573912771ad708 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:18.437855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.438326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.438439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.458876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:18.460125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:18.533413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 986/1965] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:18.542521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.542814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.543123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.564489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:18.565742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:18.575407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 980/1966] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:18.580119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.590389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb85336-c229-4fa9-8691-c37df168b866 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.596071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.596692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.596972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.597243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.597896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:18.598195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.599845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.613019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 987/1967] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:18.617802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.628306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75ca6b-d091-4ee8-914b-d8973e6d4e92 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.633354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.633984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.634283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.634566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.635254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.637235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.669218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 981/1968] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:18 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:18.673517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.684552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33507682-962b-4f4a-8045-d94755229705 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.746749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.747884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.748165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.748703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.749537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.751471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.773645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 988/1969] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:18 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:18.777810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.788506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c119-83b7-4292-b282-16a7239a46cb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.794970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.795850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.796293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.796595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.797667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.800368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.812108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 982/1970] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:18.816106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.826280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78df892-2682-4cc7-9c67-a49ff00644a8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.831438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.832025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.832327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.832567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.833221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.834806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee3d54-602f-4f28-b67a-34dbc6969af0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.846169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 989/1971] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:18.850511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:18.861427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13358281-7ee4-42e6-bb3f-675ed882f9b8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:18.866554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:18.867303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.867643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.867995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] PATH_INFO: `/v3/projects/71b2aba8511a4faca003b2bc5d78d884/users/90ed2787e73743e195afe34ae55ecf38/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.884245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b2aba8511a4faca003b2bc5d78d884, user_id=90ed2787e73743e195afe34ae55ecf38, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:18.886921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:18.918191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 983/1972] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:18 2022] PUT /identity/v3/projects/71b2aba8511a4faca003b2bc5d78d884/users/90ed2787e73743e195afe34ae55ecf38/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:18.923829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea9b5-faad-41ea-bd59-b06e35c375ac None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:18.924127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea9b5-faad-41ea-bd59-b06e35c375ac None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:18.924426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea9b5-faad-41ea-bd59-b06e35c375ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:18.944232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-afeea9b5-faad-41ea-bd59-b06e35c375ac None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:18.945432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-afeea9b5-faad-41ea-bd59-b06e35c375ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:18.990752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:19.018870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 990/1973] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:19.029137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:19.030892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:19.092612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aa286-8383-4411-b589-f80fcd16bcc9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:19.096377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:19.096977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:19.097259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:19.097527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:19.106634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:19.108179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:19.117826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 984/1974] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:19.785524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:19.786042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:19.786248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:19.818564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] MFA Rules not processed for user `90ed2787e73743e195afe34ae55ecf38`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:19.819812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:19.893379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 991/1975] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:19.897718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:19.908439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f527e-0579-4326-8e99-c1da1f134a03 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:19.913507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:19.914355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:19.914826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:19.915330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:19.916312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:19.918356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:19.954364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 985/1976] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:19 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:19.958482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:19.971073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23612dc9-b6df-416a-ac0f-f5f3e6cf8818 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:20.053460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:20.054077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.054361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.054636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:20.055325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:20.057302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:20.080377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 992/1977] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:19 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:20.084308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:20.094662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff99c2-e575-4a29-b1ba-131e0f98a13f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:20.099289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:20.099889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.100171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.100443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:20.101085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:20.102695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:20.113342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 986/1978] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:20.117988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:20.129085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbc69e-731e-4ff3-8e2b-9ba2b7422f16 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:20.134279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:20.134929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.135241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.135517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] PATH_INFO: `/v3/projects/977cd750cd7541bd83b6a9633826d0c1/users/31323187b82546d99fc89efc357e8f46/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:20.152550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] RBAC: Authorizing `identity:create_grant(project_id=977cd750cd7541bd83b6a9633826d0c1, user_id=31323187b82546d99fc89efc357e8f46, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:20.155927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:20.172496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 993/1979] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:20 2022] PUT /identity/v3/projects/977cd750cd7541bd83b6a9633826d0c1/users/31323187b82546d99fc89efc357e8f46/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:20.176627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:20.187776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d4301-bf38-4918-984a-c663f23519eb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:20.260388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:20.260998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.261275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.261545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:20.262185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:20.264175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b960-d0bf-40a2-84df-53d47e5a2bfc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:20.270798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:20.282513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 987/1980] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:20.286933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:20.298721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7441f-dcec-4c46-a0e7-5bda1117961b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:20.303376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:20.304027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.304298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.304565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] PATH_INFO: `/v3/projects/977cd750cd7541bd83b6a9633826d0c1/users/31323187b82546d99fc89efc357e8f46/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:20.306421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] RBAC: Authorizing `identity:create_grant(project_id=977cd750cd7541bd83b6a9633826d0c1, user_id=31323187b82546d99fc89efc357e8f46, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:20.309065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:20.320138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 994/1981] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:20 2022] PUT /identity/v3/projects/977cd750cd7541bd83b6a9633826d0c1/users/31323187b82546d99fc89efc357e8f46/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:20.985582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:20.985582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:20.985582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:21.060786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] MFA Rules not processed for user `31323187b82546d99fc89efc357e8f46`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:21.062138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:21.190565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 988/1982] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:21.203124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:21.205010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:21.250993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:21.320746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a519e-6a94-4b94-9364-4e2120e2804d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:21.324970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:21.325934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:21.326316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:21.326755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:21.393092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:21.394747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:21.406224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 995/1983] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:23.938745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:23.941659 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:23.958483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296da4a-0706-47da-b3e4-8e93e38b6ee3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:23.963757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:23.965122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:23.965563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:23.965835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:23.981389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:23.983891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7303a6-681d-41f6-876f-e1f8c27d11d7 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:23.998985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 989/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:29.188419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:29.189975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:29.261888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34356295-6371-46a1-adf5-fd4b8b112e27 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:29.267057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:29.267873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:29.268161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:29.268436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] PATH_INFO: `/v3/users/15d8abfc779d4aeca4d933edd016d3ca` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:29.269869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d8abfc779d4aeca4d933edd016d3ca)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:29.271905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:29.333292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d8abfc779d4aeca4d933edd016d3ca'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:29.355225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d8abfc779d4aeca4d933edd016d3ca'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:29.368823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d8abfc779d4aeca4d933edd016d3ca'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:29.369981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d8abfc779d4aeca4d933edd016d3ca'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:29.371554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d8abfc779d4aeca4d933edd016d3ca'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:29.377644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:29.394003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 996/1985] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:29 2022] DELETE /identity/v3/users/15d8abfc779d4aeca4d933edd016d3ca => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:29.398780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:29.399306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:29.399463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:29.421742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:29.423249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:29.492838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 990/1986] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:29.503146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:29.504583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:29.562632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820c9c8-c5e9-4cac-93f4-4cb32f2c00c4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:29.566615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:29.567251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:29.567855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:29.568135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:29.578017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:29.579522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:29.589338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 997/1987] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:29.942174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:29.943817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:29.960434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c21e7-180f-491e-9262-e32ea1a016a0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:29.967607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:29.968262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:29.968543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:29.969292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] PATH_INFO: `/v3/projects/16dda3204e1e47b88fe328c0f8df2e02` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:29.970466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16dda3204e1e47b88fe328c0f8df2e02)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:29.985868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:30.031199 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16dda3204e1e47b88fe328c0f8df2e02', 'request_id': 'req-b4712177-0f1a-4d1a-9e5d-802eb788ee75'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.037316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16dda3204e1e47b88fe328c0f8df2e02', 'request_id': 'req-b4712177-0f1a-4d1a-9e5d-802eb788ee75'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.043065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16dda3204e1e47b88fe328c0f8df2e02', 'request_id': 'req-b4712177-0f1a-4d1a-9e5d-802eb788ee75'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.044714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:30.066788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] The token cache is being invalidate because project 16dda3204e1e47b88fe328c0f8df2e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:30.067166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.067826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.069441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 991/1988] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:29 2022] DELETE /identity/v3/projects/16dda3204e1e47b88fe328c0f8df2e02 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:30.073326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:30.085504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc01a83-5e80-412a-93bd-72edc24efa85 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:30.165261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:30.165944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.166221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.166487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] PATH_INFO: `/v3/users/6a4ebd0a645c47e8b7205ff3aaa7dab6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.167757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4ebd0a645c47e8b7205ff3aaa7dab6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:30.169715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:30.221282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4ebd0a645c47e8b7205ff3aaa7dab6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.236907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4ebd0a645c47e8b7205ff3aaa7dab6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.245701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4ebd0a645c47e8b7205ff3aaa7dab6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.246472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4ebd0a645c47e8b7205ff3aaa7dab6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.247152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4f44bfdd-77f2-4f4b-8681-f3db57ec8f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4ebd0a645c47e8b7205ff3aaa7dab6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.260958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 998/1989] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:30 2022] DELETE /identity/v3/users/6a4ebd0a645c47e8b7205ff3aaa7dab6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:30.567917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:30.569607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:30.590097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4712177-0f1a-4d1a-9e5d-802eb788ee75 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:30.650530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:30.651287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.651628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.651901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] PATH_INFO: `/v3/projects/edcb5eb0fbf34555be2102e4245cb35b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.653004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edcb5eb0fbf34555be2102e4245cb35b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:30.655079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:30.704235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edcb5eb0fbf34555be2102e4245cb35b', 'request_id': 'req-3f184a33-6288-444b-bc22-0a01406e51a2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.710813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edcb5eb0fbf34555be2102e4245cb35b', 'request_id': 'req-3f184a33-6288-444b-bc22-0a01406e51a2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.716805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edcb5eb0fbf34555be2102e4245cb35b', 'request_id': 'req-3f184a33-6288-444b-bc22-0a01406e51a2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.734966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] The token cache is being invalidate because project edcb5eb0fbf34555be2102e4245cb35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:30.735281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.735873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f184a33-6288-444b-bc22-0a01406e51a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:30.737410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 992/1990] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:30 2022] DELETE /identity/v3/projects/edcb5eb0fbf34555be2102e4245cb35b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:30.742989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.743278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.743557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.763793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:30.765010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:30.779326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:30.833791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 999/1991] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:30.843521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.843984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.844429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.870950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:30.872212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:30.880601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 993/1992] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:30.888203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:30.899379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548d687-5a8b-4112-bfb6-ac1f154f4c53 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:30.904700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:30.905303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.905586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.905892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.906518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:30.907335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:30.908509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:30.924248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1000/1993] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:30.925646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:30.946987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774a7b4-9af1-4c4e-945f-6acfdd075376 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:30.953342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:30.954175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:30.954459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:30.954730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:30.955415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:30.957334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:30.993816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 994/1994] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:30 2022] POST /identity/v3/projects => generated 355 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:30.994577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.010350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cec2b-9726-4ccb-85ec-da30b2013cec admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.081603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.082207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.082781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.083055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.083687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.085673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.111238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1001/1995] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:30 2022] POST /identity/v3/users => generated 278 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:31.112812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.137117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d75a9-6702-421e-9977-0e1b518b4b3f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.144257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.144837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.145626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.145913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.146574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.148290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.163539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 995/1996] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:31.164451 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.177095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5d630-7faf-4b20-96c3-e12770e8890a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.182091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.182850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.183180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.183461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.184231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.186016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.220209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1002/1997] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:31.224633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.238251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee05642-b61e-46b5-a2b8-bc0115f73c9d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.243566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.244283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.244504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.244798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] PATH_INFO: `/v3/projects/a975743cef7a458a9a770fceb8430d55/users/2b6b8f10e7bc44428e93b519dd1c75f0/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.259920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a975743cef7a458a9a770fceb8430d55, user_id=2b6b8f10e7bc44428e93b519dd1c75f0, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.263067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed592c-000e-4236-bb6e-eb5fd4fc49ce admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.272578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.276205 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:31.282495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 996/1998] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:31 2022] PUT /identity/v3/projects/a975743cef7a458a9a770fceb8430d55/users/2b6b8f10e7bc44428e93b519dd1c75f0/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:31.293313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.293609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.293917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.298330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dc0cf-cf42-4dfd-b9fb-0d0823d980aa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.328160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] MFA Rules not processed for user `2b6b8f10e7bc44428e93b519dd1c75f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:31.329970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:31.335970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:31.378365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.379444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.379903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.380795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] PATH_INFO: `/v3/users/a45da0676d2d4546997d31583e4c28f7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.382551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45da0676d2d4546997d31583e4c28f7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.384732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.431109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 997/1999] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:31.435566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.437471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:31.463644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d798b73-db8a-4b4a-b8df-ed44aa71851b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.469646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.470267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.470544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.470821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.471473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.473479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.476329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45da0676d2d4546997d31583e4c28f7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:31.491309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45da0676d2d4546997d31583e4c28f7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:31.502494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45da0676d2d4546997d31583e4c28f7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:31.504731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45da0676d2d4546997d31583e4c28f7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:31.505639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e0466465-e1ea-49fc-93aa-769b139f4c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45da0676d2d4546997d31583e4c28f7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:31.520514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1003/2000] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:31 2022] DELETE /identity/v3/users/a45da0676d2d4546997d31583e4c28f7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:31.524928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.525221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.525496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.531611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 998/2001] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:31 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:31.536036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.541817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:31.552202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:31.553442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:31.632106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1004/2002] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:31.645174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.647218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:31.675312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838cd83-0f16-4054-a697-2e5fa9c0c8f1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.681849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.682475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.682883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.683089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.683784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.686235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.711775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 999/2003] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:31 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:31.716308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.727321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464037-0f41-4548-8f7d-6badb1dafe50 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.728586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3da3bd-6e0d-4039-912c-33315de52b79 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.731403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.732045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.732605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.732735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.734177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.734818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.735104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.735375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.736069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.737726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.742443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.743893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.748409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1000/2004] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:31.752981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.756469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1005/2005] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:31.763830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b0910-8f65-45b2-9557-ccdc7bd8245d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.768486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.769208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.769429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.769686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] PATH_INFO: `/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8/users/60a5c605607747bdbb589b7e6e00c8c0/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.784775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3df1e0b2314220af8a3fbc9e320ef8, user_id=60a5c605607747bdbb589b7e6e00c8c0, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.787482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.811894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1001/2006] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:31 2022] PUT /identity/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8/users/60a5c605607747bdbb589b7e6e00c8c0/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:31.816325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.828020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fd790-5b8c-4ac2-a82b-939a01dd6492 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.882035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:31.896842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.897453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.897725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.897989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.898623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.900758 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0326d9-3bc1-435d-9232-958954c96e10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.923739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1006/2007] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:31.928037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:31.939257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f96fa7-0d49-4ad9-8d39-30866c214ccc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:31.946183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:31.947010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.947386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.947718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] PATH_INFO: `/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8/users/60a5c605607747bdbb589b7e6e00c8c0/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:31.949740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3df1e0b2314220af8a3fbc9e320ef8, user_id=60a5c605607747bdbb589b7e6e00c8c0, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:31.953593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:31.966365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1002/2008] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:31 2022] PUT /identity/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8/users/60a5c605607747bdbb589b7e6e00c8c0/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:31.978584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:31.978872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:31.979183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.013772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] MFA Rules not processed for user `60a5c605607747bdbb589b7e6e00c8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:32.015121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.050289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.052130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.086943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:32.109203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1007/2009] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:32.120780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.122138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.128992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbb02-791f-4318-9f66-a8fc395daebb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.133892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.134619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.134814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.135116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] PATH_INFO: `/v3/projects/41fc730ba093494ca268d17dda13b370` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.136233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41fc730ba093494ca268d17dda13b370)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.138173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.188384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8aca4f-34f3-40e3-a785-e94ed61704e4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.192609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41fc730ba093494ca268d17dda13b370', 'request_id': 'req-d7e14876-8e32-4d55-83b5-14bf0343f4c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:32.203871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41fc730ba093494ca268d17dda13b370', 'request_id': 'req-d7e14876-8e32-4d55-83b5-14bf0343f4c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:32.212645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41fc730ba093494ca268d17dda13b370', 'request_id': 'req-d7e14876-8e32-4d55-83b5-14bf0343f4c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:32.229329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] The token cache is being invalidate because project 41fc730ba093494ca268d17dda13b370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:32.229758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:32.230593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7e14876-8e32-4d55-83b5-14bf0343f4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:32.236004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1003/2010] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:32 2022] DELETE /identity/v3/projects/41fc730ba093494ca268d17dda13b370 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:32.238215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6551e5-ce6c-4759-94b2-4dde3f4b91ac None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.238537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6551e5-ce6c-4759-94b2-4dde3f4b91ac None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.239382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6551e5-ce6c-4759-94b2-4dde3f4b91ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.266136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cb6551e5-ce6c-4759-94b2-4dde3f4b91ac None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:32.267740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cb6551e5-ce6c-4759-94b2-4dde3f4b91ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.288194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.289556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.290646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.290948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.292876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.355647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:32.364839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1004/2011] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:32.372753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.373063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.373337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.398473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.400154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.404605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:32.406476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.418006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1005/2012] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:32.419443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1008/2013] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:32.422614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.433319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd5d9b-8249-4439-830d-6691d1be94db None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.438601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.439258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.439983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.440260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.441410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:32.441410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.442797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.463990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1006/2014] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:07:32.465077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.486077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065a7d-9a57-4c8c-928a-b13c2b680cf8 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.504821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.505549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.505896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.506235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.506939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.508966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.533972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.536212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.540157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1009/2015] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:32 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:32.545176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.554783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff1bbd-c85b-4cb6-95fb-7b97af63d1bd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.608402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:32.641259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.641640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07797-b1d2-47c3-b5c9-a6b60d7491b6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.642006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.642474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.642937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.644740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:32.649080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.649713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.649992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.650253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.652139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.654475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.698418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1010/2016] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:32 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:32.702069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.733260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c367e-0720-4d57-95f4-95f6d9c2ff68 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.753982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.754712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.755080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.755419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.756198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.757982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.768172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.769971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.779672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1011/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:32.783330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1007/2018] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:32.783992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.798990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db65ba-5d29-40c7-af59-54da3c5eabf6 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.806999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.808082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.808296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.808571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.809220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.811100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d70c6-1b85-425d-a2d2-5fc037eebc88 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.826330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1008/2019] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:32.833944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:32.845707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dc8e8-833c-444b-a3f5-7739da960b90 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:32.851900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:32.852761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.853047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.854285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] PATH_INFO: `/v3/projects/75af9ee79a5a4861a127837e0129e33d/users/02dd9e622c6540fc94e9ebec19c626ce/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.887707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75af9ee79a5a4861a127837e0129e33d, user_id=02dd9e622c6540fc94e9ebec19c626ce, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:32.887707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:32.911082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1012/2020] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:32 2022] PUT /identity/v3/projects/75af9ee79a5a4861a127837e0129e33d/users/02dd9e622c6540fc94e9ebec19c626ce/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:32.920793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:32.921222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:32.921580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:32.965027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] MFA Rules not processed for user `02dd9e622c6540fc94e9ebec19c626ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:32.965400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:33.088929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1009/2021] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:33.103086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:33.105068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:33.166556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006021-4555-407d-b373-25f8982bca94 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:33.176584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:33.177934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:33.179523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:33.180061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:33.194341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:33.197883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b3808-5360-4eda-a61f-0dd80985d02e service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:33.212154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1013/2022] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:36.549151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:36.624614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380503e-87a3-44d8-996b-b38b053b951b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:36.629555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:36.630192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:36.630464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:36.630732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] PATH_INFO: `/v3/users/02dd9e622c6540fc94e9ebec19c626ce` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:36.631915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dd9e622c6540fc94e9ebec19c626ce)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:36.633837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:36.649129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:36.703423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dd9e622c6540fc94e9ebec19c626ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:36.704142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dd9e622c6540fc94e9ebec19c626ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:36.712413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dd9e622c6540fc94e9ebec19c626ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:36.720808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dd9e622c6540fc94e9ebec19c626ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:36.721692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dd9e622c6540fc94e9ebec19c626ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:36.750101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1010/2023] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:36 2022] DELETE /identity/v3/users/02dd9e622c6540fc94e9ebec19c626ce => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:36.754571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:36.755181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:36.755181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:36.782666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:36.784043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:36.795798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:36.865685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1014/2024] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:36.875906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:36.877332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:36.954198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cae19b-28b9-4490-944a-e6065f6b680b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:36.958338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:36.958958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:36.959286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:36.959543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:36.972818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:36.974418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0564c-f249-46be-a6db-a30394579403 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:36.991344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1011/2025] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:37.381524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:37.386849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:37.408411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574e111-4832-4104-ba08-31e0847efbe3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:37.433123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:37.438256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.439019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.441615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] PATH_INFO: `/v3/projects/75af9ee79a5a4861a127837e0129e33d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.442529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75af9ee79a5a4861a127837e0129e33d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:37.445874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:37.556426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75af9ee79a5a4861a127837e0129e33d', 'request_id': 'req-c69177bf-864f-4688-ab63-8b7b3c9f80e9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:37.569615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75af9ee79a5a4861a127837e0129e33d', 'request_id': 'req-c69177bf-864f-4688-ab63-8b7b3c9f80e9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:37.577109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75af9ee79a5a4861a127837e0129e33d', 'request_id': 'req-c69177bf-864f-4688-ab63-8b7b3c9f80e9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:37.598320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] The token cache is being invalidate because project 75af9ee79a5a4861a127837e0129e33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:37.598484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:37.599331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c69177bf-864f-4688-ab63-8b7b3c9f80e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:37.601097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1015/2026] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:37 2022] DELETE /identity/v3/projects/75af9ee79a5a4861a127837e0129e33d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:37.606728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.607810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.608085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.663261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:37.664556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:37.753653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1012/2027] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:37.762014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.762547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.762642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.783713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:37.784940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:37.793035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1016/2028] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:37.798404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:37.809703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003d5e9-bd44-4768-af2d-d099d30198a7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:37.814757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:37.815400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.815681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.815955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.816613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:37.816910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:37.818582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:37.830486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1013/2029] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:37.835126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:37.844874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9747676-52d2-4a76-8ac4-467608d8809c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:37.849517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:37.850116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.850394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.850665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.851339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:37.853242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:37.887992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1017/2030] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:37 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:37.894719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:37.906507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5373-6c51-46ce-a10d-96d0ab5c9231 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:37.926471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:37.968813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:37.969423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:37.969703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:37.969974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:37.970682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:37.972560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:38.009640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1014/2031] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:37 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:38.010865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:38.022615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ecbcc-23cd-4afe-89c9-449b0c096fed admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:38.027485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:38.028079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:38.028363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:38.028631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:38.029543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:38.031258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:38.042700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1018/2032] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:38.049075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:38.062579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421f08e-51ea-4dd6-9c9c-eb0b05a147f9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:38.067901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:38.068531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:38.068808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:38.069083 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:38.069793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:38.071510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aba1bd-ce05-47c8-a58d-d9bea2429946 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:38.086216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1015/2033] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:38.090812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:38.101453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07199fd-33d1-473d-8a33-90738a9215dd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:38.106927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:38.108090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:38.108430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:38.108780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] PATH_INFO: `/v3/projects/d362e24b36234e6b9a62f07a06dcd9f3/users/ffdeb3e2f0ec411682815e1d4197f662/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:38.126356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d362e24b36234e6b9a62f07a06dcd9f3, user_id=ffdeb3e2f0ec411682815e1d4197f662, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:38.129373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:38.142795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1019/2034] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:38 2022] PUT /identity/v3/projects/d362e24b36234e6b9a62f07a06dcd9f3/users/ffdeb3e2f0ec411682815e1d4197f662/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:38.151546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbfe50-2b84-4698-8c25-ebced60ae704 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:38.151732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbfe50-2b84-4698-8c25-ebced60ae704 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:38.152188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbfe50-2b84-4698-8c25-ebced60ae704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:38.173140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-52bbfe50-2b84-4698-8c25-ebced60ae704 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:38.174397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-52bbfe50-2b84-4698-8c25-ebced60ae704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:38.239440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1016/2035] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:38.249953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:38.251991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:38.285235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:38.315265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0395b-b54f-49ee-91a4-8985d1f56b92 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:38.320547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:38.321438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:38.321830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:38.322216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:38.333868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:38.335792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:38.347301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1020/2036] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:39.292813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c9a3d-8a44-4f56-b2d4-300a2c883904 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:39.293850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c9a3d-8a44-4f56-b2d4-300a2c883904 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:39.294123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c9a3d-8a44-4f56-b2d4-300a2c883904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:39.323649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cc4c9a3d-8a44-4f56-b2d4-300a2c883904 None None] MFA Rules not processed for user `ffdeb3e2f0ec411682815e1d4197f662`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:39.324874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cc4c9a3d-8a44-4f56-b2d4-300a2c883904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:39.399116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1017/2037] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:39.410458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:39.412292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:39.494209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b9d1a-b42c-4de7-be85-eb44008ea32a service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:39.499624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:39.500231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:39.500659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:39.500981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:39.510117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:39.511605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92af884-7c44-4ab8-9215-d464066cba33 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:39.520514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1021/2038] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:40.949484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986c7d-64c3-437e-ac05-0e94344aa983 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:40.950094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986c7d-64c3-437e-ac05-0e94344aa983 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:40.950094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986c7d-64c3-437e-ac05-0e94344aa983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:40.975799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-69986c7d-64c3-437e-ac05-0e94344aa983 None None] MFA Rules not processed for user `ffdeb3e2f0ec411682815e1d4197f662`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:40.975799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-69986c7d-64c3-437e-ac05-0e94344aa983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:40.983081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1018/2039] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:41.088605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ad6b-4975-4f08-85b2-e5582a657902 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:41.089147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ad6b-4975-4f08-85b2-e5582a657902 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:41.089288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5ad6b-4975-4f08-85b2-e5582a657902 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:41.090644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1022/2040] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:07:41.355758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa6bb-58f6-4d85-8ab4-80418ac53306 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:41.356084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa6bb-58f6-4d85-8ab4-80418ac53306 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:41.356904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa6bb-58f6-4d85-8ab4-80418ac53306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:41.384209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4e3fa6bb-58f6-4d85-8ab4-80418ac53306 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:41.385922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4e3fa6bb-58f6-4d85-8ab4-80418ac53306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:41.415336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:41.477771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1019/2041] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:07:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:41.489636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100b55-09d7-4541-bda8-8a958a376dc4 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:41.489913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100b55-09d7-4541-bda8-8a958a376dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:41.490191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100b55-09d7-4541-bda8-8a958a376dc4 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:41.491612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1023/2042] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 31 04:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:07:41.498966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90e97-c8d7-4aa0-9fa0-b9a92bf9a8d8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:41.499284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90e97-c8d7-4aa0-9fa0-b9a92bf9a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:41.499558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae90e97-c8d7-4aa0-9fa0-b9a92bf9a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:41.521145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-aae90e97-c8d7-4aa0-9fa0-b9a92bf9a8d8 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:41.523155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aae90e97-c8d7-4aa0-9fa0-b9a92bf9a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:41.532480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1020/2043] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 31 04:07:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:41.540031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11100b55-09d7-4541-bda8-8a958a376dc4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:41.555531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11100b55-09d7-4541-bda8-8a958a376dc4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:41.565603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:41.566397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:41.566539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:41.566790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:41.576450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:41.578131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115989b-c1e5-4e1f-812e-78c01e2d5ea8 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:41.596485 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1024/2044] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:42.981533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:42.982148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:42.982253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:42.983655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1021/2045] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:07:43.004876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:43.005932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:43.006173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:43.038486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] MFA Rules not processed for user `bd2e18144a794d36b3d81cd3c8ef9cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:43.040281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:43.055349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1025/2046] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:07:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:43.075177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:43.077843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:43.105496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93805-0791-4d97-b463-15f65a0e66c2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:43.112640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:43.114439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:43.114960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:43.115376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:43.135667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:43.138542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:43.140845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:43.145635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:43.154517 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1022/2047] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:43.165384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e67a-9b05-4531-ae5c-a4bc073c7937 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:43.173212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:43.173886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:43.174163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:43.174432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] PATH_INFO: `/v3/users/90ed2787e73743e195afe34ae55ecf38` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:43.175929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ed2787e73743e195afe34ae55ecf38)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:43.178015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:43.253833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ed2787e73743e195afe34ae55ecf38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.277812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ed2787e73743e195afe34ae55ecf38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.291697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ed2787e73743e195afe34ae55ecf38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.291868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ed2787e73743e195afe34ae55ecf38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.293737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ed2787e73743e195afe34ae55ecf38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.311660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1026/2048] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:43 2022] DELETE /identity/v3/users/90ed2787e73743e195afe34ae55ecf38 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:43.720482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:43.743636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce04b3-d33f-402e-8628-de9081bade43 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:43.859902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:43.860539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:43.860817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:43.861086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] PATH_INFO: `/v3/projects/71b2aba8511a4faca003b2bc5d78d884` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:43.862205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b2aba8511a4faca003b2bc5d78d884)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:43.864478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:43.916894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b2aba8511a4faca003b2bc5d78d884', 'request_id': 'req-10704802-6932-47ee-93b4-0b7b49f2e7c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.924849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b2aba8511a4faca003b2bc5d78d884', 'request_id': 'req-10704802-6932-47ee-93b4-0b7b49f2e7c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.931831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b2aba8511a4faca003b2bc5d78d884', 'request_id': 'req-10704802-6932-47ee-93b4-0b7b49f2e7c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.946186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] The token cache is being invalidate because project 71b2aba8511a4faca003b2bc5d78d884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:43.946498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.947145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:43.948955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1023/2049] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:43 2022] DELETE /identity/v3/projects/71b2aba8511a4faca003b2bc5d78d884 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:43.953449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:43.954968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:43.969170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea757d3-3964-4335-911e-6d0e5f2c0a8a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:43.993817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:44.034777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:44.035518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.035737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.036100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] PATH_INFO: `/v3/users/31323187b82546d99fc89efc357e8f46` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.037250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31323187b82546d99fc89efc357e8f46)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:44.039322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:44.087938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31323187b82546d99fc89efc357e8f46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.108054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31323187b82546d99fc89efc357e8f46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.116097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31323187b82546d99fc89efc357e8f46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.116949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31323187b82546d99fc89efc357e8f46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.117561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-145d509d-1fba-4ad7-b001-eeaca63ab1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31323187b82546d99fc89efc357e8f46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.139972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1027/2050] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:43 2022] DELETE /identity/v3/users/31323187b82546d99fc89efc357e8f46 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:44.360846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:44.362939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:44.378373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10704802-6932-47ee-93b4-0b7b49f2e7c4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:44.415902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:44.457762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:44.459147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.459638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.460112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] PATH_INFO: `/v3/projects/977cd750cd7541bd83b6a9633826d0c1` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.461529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977cd750cd7541bd83b6a9633826d0c1)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:44.463989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:44.545342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977cd750cd7541bd83b6a9633826d0c1', 'request_id': 'req-57728c9f-8a65-47d0-8c35-6470a554d8e7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.558223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977cd750cd7541bd83b6a9633826d0c1', 'request_id': 'req-57728c9f-8a65-47d0-8c35-6470a554d8e7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.578313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977cd750cd7541bd83b6a9633826d0c1', 'request_id': 'req-57728c9f-8a65-47d0-8c35-6470a554d8e7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.597287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] The token cache is being invalidate because project 977cd750cd7541bd83b6a9633826d0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:44.598011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.599196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-57728c9f-8a65-47d0-8c35-6470a554d8e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:44.611072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.611072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.611072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.612834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1024/2051] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:44 2022] DELETE /identity/v3/projects/977cd750cd7541bd83b6a9633826d0c1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:44.655244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:44.656556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:44.784381 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1028/2052] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:44.791942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.792222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.792495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.833436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:44.834934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:44.862273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1025/2053] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:44.867130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:44.893300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300530d9-b3af-44ba-8051-4f6fcfd6850b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:44.898546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:44.899425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.899537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.899741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.900615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:44.901159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:44.906565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:44.929107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1029/2054] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:44.934388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:44.947139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a957c0a-6303-4f0a-ac05-edcef1ebdd4b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:44.967901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:44.968525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:44.968801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:44.969060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:44.969711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:44.971694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.032018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1026/2055] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:44 2022] POST /identity/v3/projects => generated 361 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:45.036723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.051773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6be8b-358c-428d-a66f-417ff3cd22b5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.093833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:45.136501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.137447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.138766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.139664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.140492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.143871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.174648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1030/2056] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:45 2022] POST /identity/v3/users => generated 281 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:45.178846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.192948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d306-6c1b-4f93-b407-6672c10049ef admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.200316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.202159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.202510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.202849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.203792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.206493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.223059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1027/2057] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:45.226049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.239751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caadf3-52ce-4a0f-9759-ff8b3c038532 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.245949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.247083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.247577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.248967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.249812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.252709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d488b-c703-4cab-a92c-4d69b485ca99 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.279093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1031/2058] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:45.287457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.300093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528a3fe-5e2a-417e-acc2-f554ff8ba0ad admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.306391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.307621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.308071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.308321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] PATH_INFO: `/v3/projects/6d56ae90fe81422190ef24bad5b0cd5d/users/a8893becdb4540ec879b0e04a4dbfcd4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.330882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d56ae90fe81422190ef24bad5b0cd5d, user_id=a8893becdb4540ec879b0e04a4dbfcd4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.333991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.357166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1028/2059] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:45 2022] PUT /identity/v3/projects/6d56ae90fe81422190ef24bad5b0cd5d/users/a8893becdb4540ec879b0e04a4dbfcd4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:45.361715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.362007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.362303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.431851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] MFA Rules not processed for user `a8893becdb4540ec879b0e04a4dbfcd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:45.433274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:45.518885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1032/2060] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:45.557574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.563222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:45.631781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b64dc-c1fe-45fd-a220-e9615eccf63a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.635448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.636128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.636471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.636823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.646095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.647677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5b895-4b10-49ec-8933-cb002376e659 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.651482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:45.658267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1029/2061] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:45.675261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.744744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0db9-fd76-462b-9ca5-b8b77caa909a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:45.749268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:45.749896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.750169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.750437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] PATH_INFO: `/v3/users/a8893becdb4540ec879b0e04a4dbfcd4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.751597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8893becdb4540ec879b0e04a4dbfcd4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:45.753497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:45.801207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:45.804556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8893becdb4540ec879b0e04a4dbfcd4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:45.819583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8893becdb4540ec879b0e04a4dbfcd4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:45.826754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8893becdb4540ec879b0e04a4dbfcd4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:45.827794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8893becdb4540ec879b0e04a4dbfcd4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:45.828733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8893becdb4540ec879b0e04a4dbfcd4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:45.845542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1033/2062] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:45 2022] DELETE /identity/v3/users/a8893becdb4540ec879b0e04a4dbfcd4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:45.850025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:45.851276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:45.851784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:45.875638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:45.877078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:45.947151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1030/2063] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:45.960434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:45.962846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:46.028360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849af9-fec6-4c9b-bc08-716dd2e4bb2a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:46.032941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:46.033556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.033838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.034275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.046105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:46.048620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dfd6e-de41-4b4d-a5b8-6fd8212767c2 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:46.062801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1034/2064] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:46.487009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:46.488917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:46.515150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510fb0b-aae9-40b1-b939-4ce97e4bc7a0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:46.521737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:46.522341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.522557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.522836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] PATH_INFO: `/v3/projects/6d56ae90fe81422190ef24bad5b0cd5d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.525809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d56ae90fe81422190ef24bad5b0cd5d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:46.528979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:46.596351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d56ae90fe81422190ef24bad5b0cd5d', 'request_id': 'req-159bda34-85c2-4dbd-b30d-89fa1b645d86'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:46.604380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d56ae90fe81422190ef24bad5b0cd5d', 'request_id': 'req-159bda34-85c2-4dbd-b30d-89fa1b645d86'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:46.610902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d56ae90fe81422190ef24bad5b0cd5d', 'request_id': 'req-159bda34-85c2-4dbd-b30d-89fa1b645d86'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:46.630624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] The token cache is being invalidate because project 6d56ae90fe81422190ef24bad5b0cd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:46.631202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:46.632079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159bda34-85c2-4dbd-b30d-89fa1b645d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:46.636953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1031/2065] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:46 2022] DELETE /identity/v3/projects/6d56ae90fe81422190ef24bad5b0cd5d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:46.639328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.639979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.640847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.663905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:46.665496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:46.747153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1035/2066] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:46.748974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.749254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.749585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.773674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:46.776087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:46.788746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1032/2067] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:46.793626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:46.838583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bb03d-c89f-4549-ae48-b79c1266901d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:46.860091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:46.861145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.861282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.861612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.862139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:46.862463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:46.864488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:46.882589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1036/2068] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:46.883803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:46.894160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc6e2d-00d2-4050-b62d-b083586b8823 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:46.898909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:46.899595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:46.899880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:46.900151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:46.900793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:46.902861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:46.935580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1033/2069] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:46 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:46.939505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:46.956923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e685f3-db31-45d5-9541-54f3bbda710c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.030386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:47.043670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.044842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.045380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.046078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.047148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.049212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.074461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1037/2070] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:46 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:47.076085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.086832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34935e9e-857e-41a4-bb8c-3e21bcf1f9ae admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.091717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.092308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.092579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.092844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.093490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.095176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.110155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1034/2071] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:47.111841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.124970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c672b3-6963-4a5f-af69-f6eb1df85699 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.139247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.139917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.140199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.140470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.141128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.143254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e871c3-ae87-4e7a-bc9c-b067da27cd15 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.160801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1038/2072] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:47.165023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.180484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca1ec4-3dc1-46b1-9211-f17a4479430a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.185468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.186146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.186458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.186756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] PATH_INFO: `/v3/projects/ed67db736b2c4495b0190d8ce54fa26a/users/0baaf929f3094483971c3f27abeec50b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.204542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed67db736b2c4495b0190d8ce54fa26a, user_id=0baaf929f3094483971c3f27abeec50b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.208421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.227283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1035/2073] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:47 2022] PUT /identity/v3/projects/ed67db736b2c4495b0190d8ce54fa26a/users/0baaf929f3094483971c3f27abeec50b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:47.239256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.239542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.239830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.273565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] MFA Rules not processed for user `0baaf929f3094483971c3f27abeec50b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:47.274722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:47.341869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1039/2074] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:47.346507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.369211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:47.418543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb6cef-8e94-4662-80bf-239cac258e9b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.423198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.423807 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.424084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.424360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.424994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.427159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.465095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1036/2075] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:47 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:47.465438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.491893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc0be8-9adc-47c4-9e3c-051ba4e94928 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.522990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.527501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:47.598387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.599137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.599511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.599695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.600370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.613375 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.623266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b6012-45a8-4b7d-be90-b0b8d2af43fd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.630226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.631019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.631402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.631833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] PATH_INFO: `/v3/users/7103b8b46630401889a8e040c0e9ad62` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.635500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7103b8b46630401889a8e040c0e9ad62)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.639219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.669906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1040/2076] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:47 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:47.675188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.688073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0ea97-4a2e-44da-a324-e689971801fd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.693543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.694361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.694859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.695307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.696207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.698037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.709647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1041/2077] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:47.713567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.724390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5e43a-8d7a-4b16-9aaf-464863242a66 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.728964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7103b8b46630401889a8e040c0e9ad62'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:47.730149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.730280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7103b8b46630401889a8e040c0e9ad62'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:47.731214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.732454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.733674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.735145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.739833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7103b8b46630401889a8e040c0e9ad62'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:47.743721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.752933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7103b8b46630401889a8e040c0e9ad62'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:47.754033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-608a3f9c-97f1-42f8-acd8-4f6855205e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7103b8b46630401889a8e040c0e9ad62'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:47.759539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1042/2078] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:47.763728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.766030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:47.779957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1037/2079] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:47 2022] DELETE /identity/v3/users/7103b8b46630401889a8e040c0e9ad62 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:47.792068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa29905-3ee6-408b-81e8-db156652df56 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:47.854025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:47.897528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:47.898346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.898624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.898890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] PATH_INFO: `/v3/projects/738d024367394821a41cfb36714b00b4/users/e2ed145326ec4d54a1268128c6b2f5e1/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.917986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738d024367394821a41cfb36714b00b4, user_id=e2ed145326ec4d54a1268128c6b2f5e1, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:47.920856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:47.940694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1043/2080] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:47 2022] PUT /identity/v3/projects/738d024367394821a41cfb36714b00b4/users/e2ed145326ec4d54a1268128c6b2f5e1/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:47.950721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:47.951092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:47.951376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:47.983071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:47.985155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.015831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] MFA Rules not processed for user `e2ed145326ec4d54a1268128c6b2f5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:48.017094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.053263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:48.065824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0865f86-571c-4efe-9744-c8fd075b2406 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.072497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.073152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.073420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.073687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] PATH_INFO: `/v3/projects/dfea8632ec6f4cb5ac84e58d4fe10e6a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.074820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfea8632ec6f4cb5ac84e58d4fe10e6a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.077208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.139172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfea8632ec6f4cb5ac84e58d4fe10e6a', 'request_id': 'req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:48.139748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1038/2081] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:48.140889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.142091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfea8632ec6f4cb5ac84e58d4fe10e6a', 'request_id': 'req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:48.148332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfea8632ec6f4cb5ac84e58d4fe10e6a', 'request_id': 'req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:48.151941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59957-6dc1-402d-ace3-de414edba8cc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.162879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] The token cache is being invalidate because project dfea8632ec6f4cb5ac84e58d4fe10e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:48.163202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:48.164272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ef08feb9-2aa8-4a8b-b6ff-40280312ba33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:48.170141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1044/2082] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:47 2022] DELETE /identity/v3/projects/dfea8632ec6f4cb5ac84e58d4fe10e6a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:48.171757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee910-c0b6-4803-80e5-4a74136e0f6b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.172061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee910-c0b6-4803-80e5-4a74136e0f6b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.172336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee910-c0b6-4803-80e5-4a74136e0f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.194146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-275ee910-c0b6-4803-80e5-4a74136e0f6b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:48.195395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-275ee910-c0b6-4803-80e5-4a74136e0f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.220693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.221298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.221577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.221850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.222482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.224443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.229176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:48.253684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1039/2083] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:48.257779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.258368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1045/2084] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:48.259560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.267136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.267507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.267840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.290628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:48.291867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.332837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a165a3a-bc16-493f-b983-5358da8ff95d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.337974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.338601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.338874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.339311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.339974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.342357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.356109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1046/2085] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:48.361061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.367352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1040/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:48.371243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.372782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbca5f4-a314-4928-a17b-59fd5536d6c7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.378463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.379216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.379449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.379713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.380359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:48.380665 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.381532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fb346-0751-4d0b-953a-2089a28c4cf7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.382256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.386551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.387170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.387454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.387727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.388452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.390087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.398548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1047/2087] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:48.399538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.405250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1041/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:48.406412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.410324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff807a41-a411-449c-86bb-528f178c4a10 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.415683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.416301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.416597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.416855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.417513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.418273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd2b06-ec04-4270-a14c-06199481a124 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.419598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.423627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.424269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.424560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.424821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] PATH_INFO: `/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5/users/69039f1710b54cf9a8d7a75d39b4eee4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.438212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:48.441542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08415deb4b14b47bc8713f9f6a29bd5, user_id=69039f1710b54cf9a8d7a75d39b4eee4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.445032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.450683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1048/2089] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:48.455118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.459918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1042/2090] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:48 2022] PUT /identity/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5/users/69039f1710b54cf9a8d7a75d39b4eee4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:48.463884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.521203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88fc0-0c4d-427b-a0ee-138085710a87 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.526079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.526690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.526969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.527313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.527513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496d190-9973-4cc1-91d4-daab7ce412e6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.528005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.530256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.533016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.533715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.534000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.534271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.534954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.536676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.549011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1043/2091] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:48.552901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1049/2092] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:48.553074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.556889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.566480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768fb9-a59a-4ad7-9a80-8fbeb584df22 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.567227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc029-cec1-4ec7-91f2-34a67fd43a82 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.571701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.572041 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.572562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.572672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.572839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.572953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.573173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] PATH_INFO: `/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5/users/69039f1710b54cf9a8d7a75d39b4eee4/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.573294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.573797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.575183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08415deb4b14b47bc8713f9f6a29bd5, user_id=69039f1710b54cf9a8d7a75d39b4eee4, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.575459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.577951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e318f5b-da3b-4c7f-aa7d-a0243e9c4131 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.586067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1050/2093] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:48.589024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1044/2094] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:48 2022] PUT /identity/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5/users/69039f1710b54cf9a8d7a75d39b4eee4/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:48.589968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.599075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.599338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.599636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.600099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4319549-3bec-4e00-a4b3-907b6a500a23 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.627069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:48.629791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] MFA Rules not processed for user `69039f1710b54cf9a8d7a75d39b4eee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:48.631017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.667008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.667767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.668047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.668341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.668988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.670886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.681366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1051/2095] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:48.686662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.696475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37219c-4394-4a31-9b42-304fa54f381a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.701099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.701745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.702030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.702301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] PATH_INFO: `/v3/projects/cb862d8283654f4c9d6df05093dcc2a5/users/c18e77f425834df2a63b54337d032350/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.712526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1045/2096] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:48.718506 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb862d8283654f4c9d6df05093dcc2a5, user_id=c18e77f425834df2a63b54337d032350, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.721268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3814f-cfb0-499e-bd31-1f026f807c21 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.730722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.733261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.735553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1052/2097] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:48 2022] PUT /identity/v3/projects/cb862d8283654f4c9d6df05093dcc2a5/users/c18e77f425834df2a63b54337d032350/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:48.745888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.746174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.746448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.774479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] MFA Rules not processed for user `c18e77f425834df2a63b54337d032350`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:48.775813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.793299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9cd20-60de-47ea-bd3d-599a4f41384f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.808514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:48.858513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.859812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1053/2098] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:48.860502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.860848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.861184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.864351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.871257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:48.873278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:48.892811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee25807-1767-4218-aa5f-6c5d8ca305da None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:48.896860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:48.897466 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:48.897740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:48.898010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:48.907226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.908690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.917898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1054/2099] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:48.935764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:48.935912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda0fd3-cf4e-43a4-8504-6f642df3287e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:48.946932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1046/2100] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:49.667487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:49.669082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:49.733220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20ae0-5c4a-4789-952e-decd927d1733 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:49.740653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:49.741449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:49.741730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:49.741996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] PATH_INFO: `/v3/users/2b6b8f10e7bc44428e93b519dd1c75f0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:49.743231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6b8f10e7bc44428e93b519dd1c75f0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:49.745289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:49.759342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:49.802922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6b8f10e7bc44428e93b519dd1c75f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:49.818504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6b8f10e7bc44428e93b519dd1c75f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:49.826009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6b8f10e7bc44428e93b519dd1c75f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:49.826774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6b8f10e7bc44428e93b519dd1c75f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:49.827541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6b8f10e7bc44428e93b519dd1c75f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:49.844530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1055/2101] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:49 2022] DELETE /identity/v3/users/2b6b8f10e7bc44428e93b519dd1c75f0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:49.849246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:49.849714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:49.849877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:49.871221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:49.872431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:49.940913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1047/2102] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:49.952482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:49.954129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:50.020313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce2dd-a591-4080-ab44-7ecabe7046cf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:50.024256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:50.024870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:50.025145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:50.025415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:50.036503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:50.038167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:50.048532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1056/2103] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:50.407280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:50.408768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:50.426516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007a42d-bb89-4bf1-bc56-facf5e6530fb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:50.437766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:50.438698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:50.438982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:50.439391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] PATH_INFO: `/v3/projects/a975743cef7a458a9a770fceb8430d55` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:50.440598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a975743cef7a458a9a770fceb8430d55)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:50.442649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:50.496328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a975743cef7a458a9a770fceb8430d55', 'request_id': 'req-143c5e2d-c31c-4d56-8b54-42b71ce2b428'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.503190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a975743cef7a458a9a770fceb8430d55', 'request_id': 'req-143c5e2d-c31c-4d56-8b54-42b71ce2b428'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.512425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a975743cef7a458a9a770fceb8430d55', 'request_id': 'req-143c5e2d-c31c-4d56-8b54-42b71ce2b428'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.515278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:50.542613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] The token cache is being invalidate because project a975743cef7a458a9a770fceb8430d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:50.543209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.544408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.546583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1048/2104] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:50 2022] DELETE /identity/v3/projects/a975743cef7a458a9a770fceb8430d55 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:50.550515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:50.563018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d20f5a-3a62-4a6e-bcf4-23330b6210b0 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:50.631603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:50.632790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:50.633302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:50.633817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] PATH_INFO: `/v3/users/60a5c605607747bdbb589b7e6e00c8c0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:50.635556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a5c605607747bdbb589b7e6e00c8c0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:50.639053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:50.691444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:50.722059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a5c605607747bdbb589b7e6e00c8c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.742611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a5c605607747bdbb589b7e6e00c8c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.753165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a5c605607747bdbb589b7e6e00c8c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.757131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a5c605607747bdbb589b7e6e00c8c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.758133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a5c605607747bdbb589b7e6e00c8c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:50.791071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1057/2105] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:50 2022] DELETE /identity/v3/users/60a5c605607747bdbb589b7e6e00c8c0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:50.919537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:50.921104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:50.998624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5e2d-c31c-4d56-8b54-42b71ce2b428 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.071667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.072925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.073202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.073584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.075457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:51.158660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.160163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c38ba-dc61-4718-b0aa-68029f553265 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.171655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1049/2106] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:51.198661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.199996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:51.288815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9efd-a804-45ea-a8dc-e4d45b243a10 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.292861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.293529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.293885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.294625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] PATH_INFO: `/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.295931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3df1e0b2314220af8a3fbc9e320ef8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.298026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.350594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3df1e0b2314220af8a3fbc9e320ef8', 'request_id': 'req-21125197-790a-473a-a19c-ab7a3663e349'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:51.358063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3df1e0b2314220af8a3fbc9e320ef8', 'request_id': 'req-21125197-790a-473a-a19c-ab7a3663e349'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:51.373707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3df1e0b2314220af8a3fbc9e320ef8', 'request_id': 'req-21125197-790a-473a-a19c-ab7a3663e349'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:51.397390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] The token cache is being invalidate because project 8f3df1e0b2314220af8a3fbc9e320ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:51.398158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:51.399188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-21125197-790a-473a-a19c-ab7a3663e349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:51.402452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1058/2107] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:51 2022] DELETE /identity/v3/projects/8f3df1e0b2314220af8a3fbc9e320ef8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:51.408431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.408722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.409047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.448345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:51.449607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:51.539016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1050/2108] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:51.546860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.547282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.547683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.576453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:51.578418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:51.588484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1059/2109] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:51.593047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.600996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:51.605908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45c84f-0511-4daa-8463-20b5ae396cf7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.614351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.615351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.615699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.616070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.616911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:51.617307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.627897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.648597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1051/2110] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:07:51.654675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.669473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb9340-687e-4943-828b-4bdb8541fe33 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.677620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.678916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.679251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.679523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.680700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.683299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.725596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1060/2111] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:51 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:51.729741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.740455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97de9b-e31f-46b9-982a-0c87ce9b1091 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.822268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.822871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.823179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.823447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.824166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.826493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.854287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1052/2112] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:51 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:51.859626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.877149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6ab8e-e94c-4fab-8a0b-4dd45600da2d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.883569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.884198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.884471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.884746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.885397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.887353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.896783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.898514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:51.898971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1061/2113] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:51.902637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.909113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:51.913098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba2222-7599-4346-8049-8a29cb488931 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.919541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.920507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.921534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.922049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.923061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.925889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:51.937195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1062/2114] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:51.943486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:51.957277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a7f7-1d99-4840-8a13-ad12f37a782a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.962735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.963852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.964436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.964858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] PATH_INFO: `/v3/projects/15497c8da58840d69992681fcbc444b8/users/34f7cc3310aa44cbb9f51f54062e405a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.987249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e2d21-7403-4340-a7dc-627c8e345d58 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:51.993609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:51.994252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:51.994536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:51.994804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:51.995742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15497c8da58840d69992681fcbc444b8, user_id=34f7cc3310aa44cbb9f51f54062e405a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:51.998555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072b699-072b-432a-a8a2-f0e30da2a8a4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.003826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:52.022962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1063/2115] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:51 2022] PUT /identity/v3/projects/15497c8da58840d69992681fcbc444b8/users/34f7cc3310aa44cbb9f51f54062e405a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:52.035131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.035409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.035686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.085968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] MFA Rules not processed for user `34f7cc3310aa44cbb9f51f54062e405a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:52.087241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:52.095867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.097542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.109889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1053/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:52.166245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1064/2117] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:52.171578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.183761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c686d-ecb7-4daf-b2bf-c5405bc5fc22 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.222191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:52.271297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.271297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.271297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.271297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.272357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.276715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.310814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1054/2118] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:52 2022] POST /identity/v3/projects => generated 393 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:52.315094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.331534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710cd98-9c19-44c6-85a1-23139d704909 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.418261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.418882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.419456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.419921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.420493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.422966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.448878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1065/2119] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:52 2022] POST /identity/v3/users => generated 303 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:52.453109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.464323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12030c04-59fd-4dd8-b7f4-edadf2d5333d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.470391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.471270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.471549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.471814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.472461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.474165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.485863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1055/2120] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:52.490530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.501579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4c9e1-3f46-4532-8132-59d45ff748e9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.507213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.508111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.508492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.508739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] PATH_INFO: `/v3/projects/64e91e26727548c6992a10a056c151f3/users/580d6ece53c7403fac69f022a2688db1/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.523456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e91e26727548c6992a10a056c151f3, user_id=580d6ece53c7403fac69f022a2688db1, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.526112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.540244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1066/2121] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:52 2022] PUT /identity/v3/projects/64e91e26727548c6992a10a056c151f3/users/580d6ece53c7403fac69f022a2688db1/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:52.545511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.556728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3f008-8f8f-4d5e-951a-5873ae327be8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.619822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.622030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.622319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.623238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.623387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.625049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387774b9-b3ff-41c9-ba9c-88b36623f0f5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.638793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1056/2122] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:52.640059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.651074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cbd16-27ec-4afe-932c-836d7757309e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.656672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.657274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.657591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.657815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] PATH_INFO: `/v3/projects/64e91e26727548c6992a10a056c151f3/users/580d6ece53c7403fac69f022a2688db1/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.659998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e91e26727548c6992a10a056c151f3, user_id=580d6ece53c7403fac69f022a2688db1, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.662644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.675819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1067/2123] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:52 2022] PUT /identity/v3/projects/64e91e26727548c6992a10a056c151f3/users/580d6ece53c7403fac69f022a2688db1/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:52.686388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.686673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.686942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.717528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] MFA Rules not processed for user `580d6ece53c7403fac69f022a2688db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:52.718868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:52.791782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1057/2124] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:52.805150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.821002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:52.872858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b30885-158a-4d5b-b19c-16f0d74a34e0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.877216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.878093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.878498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.878899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:52.890662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:52.892339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:52.906377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1068/2125] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:52.961165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:52.963675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:52.982911 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1aa7c-3bbc-4818-92ce-49351b3ce927 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:52.989308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:52.989910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:52.990200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:52.990464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:53.017103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:53.050498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:53.051975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52329a28-8d29-495e-97e0-5c184dc96715 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:53.061362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1058/2126] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:55.960569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:55.962060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.030052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9b6d-0560-424c-96fe-f346ee18a56b service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.034713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.035378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.035657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.035928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] PATH_INFO: `/v3/users/9ee82d34660344a4a68b23cab100046e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.037136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee82d34660344a4a68b23cab100046e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.039616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.108847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee82d34660344a4a68b23cab100046e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.122674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee82d34660344a4a68b23cab100046e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.129899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee82d34660344a4a68b23cab100046e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.131070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee82d34660344a4a68b23cab100046e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.131685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee82d34660344a4a68b23cab100046e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.147085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1069/2127] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:55 2022] DELETE /identity/v3/users/9ee82d34660344a4a68b23cab100046e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:56.152177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.152764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.152892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.177644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:56.178916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.213795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.215313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.248630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1059/2128] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:56.253627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:56.258869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.260412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.298113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10f6b3-1d9d-44c7-954d-b6fe2421aae5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.303388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.304332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.304867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.305145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] PATH_INFO: `/v3/users/0baaf929f3094483971c3f27abeec50b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.306316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baaf929f3094483971c3f27abeec50b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.308341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.325374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f4cb6-a55f-4ce8-8ac9-cbe72752d27b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.329787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.330428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.330700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.331131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.342461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.344151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725170c8-c433-404d-8105-867d5a0cf725 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.355178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1060/2129] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:56.360919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baaf929f3094483971c3f27abeec50b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.374125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baaf929f3094483971c3f27abeec50b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.388608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baaf929f3094483971c3f27abeec50b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.389757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baaf929f3094483971c3f27abeec50b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.390434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baaf929f3094483971c3f27abeec50b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.407262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1070/2130] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:56 2022] DELETE /identity/v3/users/0baaf929f3094483971c3f27abeec50b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:56.412363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.412962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.412962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.433966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:56.435334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.511641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1061/2131] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:56.525079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.526518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.593927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.595383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.606748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc60a44-6188-4e3a-8c64-1ace6305421f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.609753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43265a4a-235f-4bd7-a9d5-138c14fddf0f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.612338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.612877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.613151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.613733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.615201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.615828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.616105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.616432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] PATH_INFO: `/v3/users/34f7cc3310aa44cbb9f51f54062e405a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.617541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f7cc3310aa44cbb9f51f54062e405a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.619477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.622540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:56.632579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.633429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.644625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1071/2132] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:56.681960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.683878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.704864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60934878-04c0-42b1-a2e4-39f21fa9e1c0 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:56.709924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:56.710573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.710917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.711459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] PATH_INFO: `/v3/projects/82dd303dbcd34a0198cf939cc6feda9e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.713009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82dd303dbcd34a0198cf939cc6feda9e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:56.714976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:56.730056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f7cc3310aa44cbb9f51f54062e405a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.760045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f7cc3310aa44cbb9f51f54062e405a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.771792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f7cc3310aa44cbb9f51f54062e405a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.784128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f7cc3310aa44cbb9f51f54062e405a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.785094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fcc947f3-5e0b-4737-95a2-e5702e8da38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f7cc3310aa44cbb9f51f54062e405a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.833138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82dd303dbcd34a0198cf939cc6feda9e', 'request_id': 'req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.835009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:56.845250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1062/2133] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:56 2022] DELETE /identity/v3/users/34f7cc3310aa44cbb9f51f54062e405a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:56.846496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82dd303dbcd34a0198cf939cc6feda9e', 'request_id': 'req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.850608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.851452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.852727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.873737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82dd303dbcd34a0198cf939cc6feda9e', 'request_id': 'req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.877466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:56.879286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.900076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] The token cache is being invalidate because project 82dd303dbcd34a0198cf939cc6feda9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:56.902326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.903825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6686f1-74a4-4492-a37b-4eb9b8a1a728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:56.908531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1072/2134] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:56 2022] DELETE /identity/v3/projects/82dd303dbcd34a0198cf939cc6feda9e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:56.914314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adf9c8-a439-4dab-ad58-8e3e32391b20 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.914603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adf9c8-a439-4dab-ad58-8e3e32391b20 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.928363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adf9c8-a439-4dab-ad58-8e3e32391b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.962059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-05adf9c8-a439-4dab-ad58-8e3e32391b20 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:56.964122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-05adf9c8-a439-4dab-ad58-8e3e32391b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:56.965838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1063/2135] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:56.976500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1073/2136] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:56.984148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:56.985706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:56.986911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:56.988695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:56.991290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.011225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:57.012693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.021801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1074/2137] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:57.026928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.038286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1e5cf-cd0c-457f-accc-4706ce445f9b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.044948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.046457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.047047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.047391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.048239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:57.048609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.050379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.064565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1075/2138] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:57.068985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.080335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c4525-6fdb-47b9-8159-d38660304aab admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.084886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdccb9c-7fd6-4c70-af0a-d04410ff8e07 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.085030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.085580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.085842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.086176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.087437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.089630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.090352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.090489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.090894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.093899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.102183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.103762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.122162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1064/2139] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:07:57.123184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.124938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.134558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1076/2140] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:57 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:57.138587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.139386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca1c76-25ff-4d74-b748-c7a5c7e5bce8 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.168404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:57.223801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a0f7f-88b9-469a-bcad-a4ed9c0bf429 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.228840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.229861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.230152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.230329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.230672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.230897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.231370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.231641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] PATH_INFO: `/v3/projects/ed67db736b2c4495b0190d8ce54fa26a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.231947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.233302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed67db736b2c4495b0190d8ce54fa26a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.234609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.235443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.267075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1077/2141] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:57 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:57.271181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.285497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b50d0f-9909-4ff2-b2cd-b06a9be76c86 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.293086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.293751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.294038 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.294986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.295871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.297595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.309522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1078/2142] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:57.316935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.320349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed67db736b2c4495b0190d8ce54fa26a', 'request_id': 'req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.328176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa6086-12f1-47ae-86bd-53dca386b2ea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.328935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed67db736b2c4495b0190d8ce54fa26a', 'request_id': 'req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.335688 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed67db736b2c4495b0190d8ce54fa26a', 'request_id': 'req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.351105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] The token cache is being invalidate because project ed67db736b2c4495b0190d8ce54fa26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:57.351530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.352171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.353940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1065/2143] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:57 2022] DELETE /identity/v3/projects/ed67db736b2c4495b0190d8ce54fa26a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:57.357595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.359104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.408288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.409053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.409331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.409598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.410246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.411881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.427227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1079/2144] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:57.432016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.434945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.440185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0c0b-6a20-4146-a1d6-655e3bb1dd91 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.446567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:57.447642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.448391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.448566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.451344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] PATH_INFO: `/v3/users/e2ed145326ec4d54a1268128c6b2f5e1` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.453106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ed145326ec4d54a1268128c6b2f5e1)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.453595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4eda4-dbb7-40db-9f2e-6a8b934485a7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.455500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.461568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.462947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.463942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.464111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] PATH_INFO: `/v3/projects/92e55122214e439f897e2bafd1718c44/users/9763c5457b444ab0a2486f183d25c908/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.484643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e55122214e439f897e2bafd1718c44, user_id=9763c5457b444ab0a2486f183d25c908, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.487401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.503321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1080/2145] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:57 2022] PUT /identity/v3/projects/92e55122214e439f897e2bafd1718c44/users/9763c5457b444ab0a2486f183d25c908/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:57.513529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.513529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.515902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ed145326ec4d54a1268128c6b2f5e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.516612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ed145326ec4d54a1268128c6b2f5e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.523913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ed145326ec4d54a1268128c6b2f5e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.525624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:57.533033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ed145326ec4d54a1268128c6b2f5e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.534103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c6a8b79-9e4e-4d64-a601-3a394a7da7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ed145326ec4d54a1268128c6b2f5e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.553364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1066/2146] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:57 2022] DELETE /identity/v3/users/e2ed145326ec4d54a1268128c6b2f5e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:57.555601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.555601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.555601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.586328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8bbef-9eb8-4f6c-b779-55e37d1c0f39 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.596683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] MFA Rules not processed for user `9763c5457b444ab0a2486f183d25c908`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:57.598081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.665773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.666531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.666901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.667429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] PATH_INFO: `/v3/projects/15497c8da58840d69992681fcbc444b8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.669167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15497c8da58840d69992681fcbc444b8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.672012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.682130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1067/2147] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:57.695586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.716439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.742539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15497c8da58840d69992681fcbc444b8', 'request_id': 'req-30cf03a3-732d-448b-92f9-ea67b27b8784'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.750312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15497c8da58840d69992681fcbc444b8', 'request_id': 'req-30cf03a3-732d-448b-92f9-ea67b27b8784'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.762373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15497c8da58840d69992681fcbc444b8', 'request_id': 'req-30cf03a3-732d-448b-92f9-ea67b27b8784'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.784452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] The token cache is being invalidate because project 15497c8da58840d69992681fcbc444b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:57.784751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.785406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.786990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1081/2148] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:57 2022] DELETE /identity/v3/projects/15497c8da58840d69992681fcbc444b8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:57.791426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.795798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.805453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875dd1ea-a416-44f6-ad66-6097830fdf58 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.838810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:57.877095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.878460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.879766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.880723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.882870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.886752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf03a3-732d-448b-92f9-ea67b27b8784 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.892364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.892983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.893266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.893540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] PATH_INFO: `/v3/users/580d6ece53c7403fac69f022a2688db1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:57.894775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] RBAC: Authorizing `identity:delete_user(user_id=580d6ece53c7403fac69f022a2688db1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.896869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.931531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:57.966703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:57.966703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:57.974303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '580d6ece53c7403fac69f022a2688db1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.975135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1068/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:57.975875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:57.977198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:57.992758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c992-1260-49a9-8722-eae669bc15b5 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:57.993868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '580d6ece53c7403fac69f022a2688db1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:57.997451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:57.998359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:57.998645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:57.998927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] PATH_INFO: `/v3/projects/738d024367394821a41cfb36714b00b4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.000595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738d024367394821a41cfb36714b00b4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.001667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '580d6ece53c7403fac69f022a2688db1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.002422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '580d6ece53c7403fac69f022a2688db1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.002723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.003056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '580d6ece53c7403fac69f022a2688db1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.019412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1082/2150] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:57 2022] DELETE /identity/v3/users/580d6ece53c7403fac69f022a2688db1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:07:58.020279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.021586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.071790 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738d024367394821a41cfb36714b00b4', 'request_id': 'req-f477184f-8330-46d6-91f4-f1e87e64c213'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.080292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738d024367394821a41cfb36714b00b4', 'request_id': 'req-f477184f-8330-46d6-91f4-f1e87e64c213'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.088326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738d024367394821a41cfb36714b00b4', 'request_id': 'req-f477184f-8330-46d6-91f4-f1e87e64c213'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.108732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] The token cache is being invalidate because project 738d024367394821a41cfb36714b00b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:58.109091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057944ba-3071-410a-9a42-86f6dff3e79f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.109252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.110421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.112759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1069/2151] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:57 2022] DELETE /identity/v3/projects/738d024367394821a41cfb36714b00b4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:58.117002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.118547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.192524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:58.227822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.228686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.229644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.230079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] PATH_INFO: `/v3/users/c18e77f425834df2a63b54337d032350` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.231455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18e77f425834df2a63b54337d032350)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.234904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.238000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477184f-8330-46d6-91f4-f1e87e64c213 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.244147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.245006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.245385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.245756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] PATH_INFO: `/v3/users/69039f1710b54cf9a8d7a75d39b4eee4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.247058 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69039f1710b54cf9a8d7a75d39b4eee4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.249435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.339181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18e77f425834df2a63b54337d032350'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.362068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18e77f425834df2a63b54337d032350'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.365408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69039f1710b54cf9a8d7a75d39b4eee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.366531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69039f1710b54cf9a8d7a75d39b4eee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.370099 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18e77f425834df2a63b54337d032350'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.371117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18e77f425834df2a63b54337d032350'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.371650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-da4e42f6-4efa-4afa-b366-15fe246fc540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18e77f425834df2a63b54337d032350'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.379484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69039f1710b54cf9a8d7a75d39b4eee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.382621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:58.393805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69039f1710b54cf9a8d7a75d39b4eee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.394687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69039f1710b54cf9a8d7a75d39b4eee4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.395249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1083/2152] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:58 2022] DELETE /identity/v3/users/c18e77f425834df2a63b54337d032350 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:58.396954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.398940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.401074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.420351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1070/2153] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:07:58 2022] DELETE /identity/v3/users/69039f1710b54cf9a8d7a75d39b4eee4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:58.438597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:58.440163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.477151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.478617 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.585697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fae6ba-230a-44ac-a3e0-8ed8f235674d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.590945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638620f-05ae-440b-8373-218060df131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.591395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638620f-05ae-440b-8373-218060df131a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.591558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1084/2154] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:58.591754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638620f-05ae-440b-8373-218060df131a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.595808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638620f-05ae-440b-8373-218060df131a admin admin] PATH_INFO: `/v3/projects/64e91e26727548c6992a10a056c151f3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.596789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638620f-05ae-440b-8373-218060df131a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e91e26727548c6992a10a056c151f3)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.599020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638620f-05ae-440b-8373-218060df131a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.604784 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.606557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.686038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e91e26727548c6992a10a056c151f3', 'request_id': 'req-b638620f-05ae-440b-8373-218060df131a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.699917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e91e26727548c6992a10a056c151f3', 'request_id': 'req-b638620f-05ae-440b-8373-218060df131a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.701902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28855700-d0e5-4074-9954-b64895a2bbbc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.709109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e91e26727548c6992a10a056c151f3', 'request_id': 'req-b638620f-05ae-440b-8373-218060df131a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.725668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] The token cache is being invalidate because project 64e91e26727548c6992a10a056c151f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:58.726010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.726895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b638620f-05ae-440b-8373-218060df131a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.731110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1071/2155] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:58 2022] DELETE /identity/v3/projects/64e91e26727548c6992a10a056c151f3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:58.731602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:58.734626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.734922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.735244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.760280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:58.761924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.775292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.775890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.776178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.776454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.778975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.846051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1072/2156] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:58.847458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.849275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.864577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.864854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf60a2-0c5f-4c6e-a12c-ddcfcaa631cc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.866843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457afec-6407-4ae1-a2da-6c2ca0c5ff40 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.871425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.872080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.872354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.872628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] PATH_INFO: `/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.873794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08415deb4b14b47bc8713f9f6a29bd5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.875764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.878102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:58.878237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1085/2157] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:07:58.880222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.881316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.882644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.913173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:58.914487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:58.927310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1086/2158] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:58.928418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.940608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f99bc-196b-4289-9579-6f86a06ac30a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.946870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:58.947565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:58.948089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:58.948327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:58.948881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:58.949357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:58.951187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:58.964987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08415deb4b14b47bc8713f9f6a29bd5', 'request_id': 'req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.966215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1087/2159] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:58.970924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:58.973652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08415deb4b14b47bc8713f9f6a29bd5', 'request_id': 'req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:58.980999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9387d-68bb-437c-911e-9f9c4c5765d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:58.984408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08415deb4b14b47bc8713f9f6a29bd5', 'request_id': 'req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.001866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] The token cache is being invalidate because project c08415deb4b14b47bc8713f9f6a29bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:59.002277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.003133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3d5436cf-c18e-4133-be0c-0f8ab99ef56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.005141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1073/2160] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:58 2022] DELETE /identity/v3/projects/c08415deb4b14b47bc8713f9f6a29bd5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:59.011294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6be7c-0bdc-4c84-8702-ab45a1a648db None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.011698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6be7c-0bdc-4c84-8702-ab45a1a648db None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.012048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6be7c-0bdc-4c84-8702-ab45a1a648db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.036264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fdc6be7c-0bdc-4c84-8702-ab45a1a648db None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:59.037655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fdc6be7c-0bdc-4c84-8702-ab45a1a648db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.053295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.053968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.054197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.054467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.055146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.058273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.095467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1088/2161] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:58 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:59.099879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.102294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.119511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1074/2162] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:59.127576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.127917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.128269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.151421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:59.152656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.163440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:59.227767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1075/2163] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:07:59.228866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd11dc9-ac30-443a-b876-be1066aeaaff admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.231921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.233237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.236764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.237789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.238139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.238835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.240515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.244438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.247136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b27351-a4fb-4519-ae17-547fcb9e3413 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.255777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.256706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.257085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.257537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] PATH_INFO: `/v3/projects/cb862d8283654f4c9d6df05093dcc2a5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.258893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb862d8283654f4c9d6df05093dcc2a5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.261343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.278209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1089/2164] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:07:59.279188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.299534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b27d7-9d6d-4de1-bc62-cc93cd711aab admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.305938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.307085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:59.307234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.307234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.307234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.307798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:59.308088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.310215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.325822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1090/2165] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:59.327094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.335572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb862d8283654f4c9d6df05093dcc2a5', 'request_id': 'req-bf613856-35a9-4fb6-b623-336383d7e127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.337333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf39f7-689b-4861-87dd-f01e01b6b699 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.342760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb862d8283654f4c9d6df05093dcc2a5', 'request_id': 'req-bf613856-35a9-4fb6-b623-336383d7e127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.359550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb862d8283654f4c9d6df05093dcc2a5', 'request_id': 'req-bf613856-35a9-4fb6-b623-336383d7e127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.380844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] The token cache is being invalidate because project cb862d8283654f4c9d6df05093dcc2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:07:59.381141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.385624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:07:59.395384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1076/2166] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:07:59 2022] DELETE /identity/v3/projects/cb862d8283654f4c9d6df05093dcc2a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:59.395384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.397238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.405494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.406201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.406543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.406881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.407624 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.409256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e03a3-f941-4fb1-9785-039a89e80b13 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.421646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1091/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:59.423051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.423430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.423794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.459709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:59.460853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.470395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1092/2168] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:59.477984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.477984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.487011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f297-12d9-48ca-a7f0-c555bdd7177b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.494808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf613856-35a9-4fb6-b623-336383d7e127 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.495296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.495958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.496190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.496578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.497107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.500495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca849c8a-16d4-4542-9590-ec929e4f3a91 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.500610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.501572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.501933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.502267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.503060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.505724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.521353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1093/2169] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:59.522857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.523174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.524061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.546676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1077/2170] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:59.547856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.561505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:59.562806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.629158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7dbc-9d35-4ffd-93e8-bd321e1c9193 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.636625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1094/2171] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:59.636625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.637331 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.638090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.638553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.639063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] PATH_INFO: `/v3/projects/e8da30a9a7b34fc6a0baa6021ff4eccc/users/100623a8bec74c699e7f41561b92b73b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.644806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0580-49b9-4699-8933-184f8d53859e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.650012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.650676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.651040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.651379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.652100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.654161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.656566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8da30a9a7b34fc6a0baa6021ff4eccc, user_id=100623a8bec74c699e7f41561b92b73b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.659750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.680163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1078/2172] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:59 2022] PUT /identity/v3/projects/e8da30a9a7b34fc6a0baa6021ff4eccc/users/100623a8bec74c699e7f41561b92b73b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:07:59.680518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1095/2173] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:59.681332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.684277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.704219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125824-c108-4699-aff1-b3921365c6fe admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.713660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:59.721410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:07:59.774672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6312d-649d-40b3-997f-cc336bddb1a7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.780100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.780773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.781716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.782009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.782920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.783452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.783776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.784054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.784423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.785190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900342e-c107-4926-bca6-a061205677bb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.785416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:07:59.785713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.787446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.800652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1096/2174] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:07:59.805601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1079/2175] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:07:59.805786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.805786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.805786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.806012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.817303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d559643-5311-43ea-aba7-b817c731159e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.822645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.823497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.823875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.824297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.825033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.827146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.834403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] MFA Rules not processed for user `100623a8bec74c699e7f41561b92b73b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:07:59.835823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:07:59.842108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1080/2176] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:07:59.842108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.852623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab51ff-0f6d-459c-9f6a-ad1303482884 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.859318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.860104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.860601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.861048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:07:59.862140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:07:59.865095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:07:59.914006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1081/2177] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:07:59.914006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.935420 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1097/2178] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:07:59.939916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:07:59.951435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d22707-4afc-4917-8672-deeeb045940e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.987107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9023a-46fe-42a5-a788-6caca779a77b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:07:59.992198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:07:59.992855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:07:59.993138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:07:59.993415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] PATH_INFO: `/v3/projects/4dceffdfa7224a4a84be50c94c6f3d4b/users/3fa26512830547db839cea342721da4b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.011277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dceffdfa7224a4a84be50c94c6f3d4b, user_id=3fa26512830547db839cea342721da4b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.014074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.024630 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.025245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.025578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.025852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.026516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.028460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523aac-f7b6-4e8a-af4f-372697a1878c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.028903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1082/2179] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:07:59 2022] PUT /identity/v3/projects/4dceffdfa7224a4a84be50c94c6f3d4b/users/3fa26512830547db839cea342721da4b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:00.035959 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.038404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.051448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1098/2180] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:07:59 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:00.052766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.053115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.053445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.081264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] MFA Rules not processed for user `3fa26512830547db839cea342721da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:00.082572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.099750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:00.100544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99f513-b6ac-4cfb-be4c-6749d7e88e12 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.104738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.105341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.105650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.105904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.125235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:00.158798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1099/2181] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:00.159633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.170695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.172312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.181458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1083/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:00.182255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.183658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.195451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace32a5-807d-45ab-afa2-38c0679a2fac service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.203122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.203926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.204268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.204596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.217122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.218853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.228415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1084/2183] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:00.229210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.234088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f309da-bbd6-4d04-abaa-5c15531ce856 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.239210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.239814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.239978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.240115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2cbb8-0a13-4e56-a6f6-002cba6b9728 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.240378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.241077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.242654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.244862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.245468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.245739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.246025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] PATH_INFO: `/v3/users/100623a8bec74c699e7f41561b92b73b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.247191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100623a8bec74c699e7f41561b92b73b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.249151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.253160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1100/2184] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:00.257752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.270018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff57d5-2d37-4bbb-b810-7bdcec622ede admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.274841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.275470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.275747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.276206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.276860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.278597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.290637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1101/2185] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:00.294315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.305016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf386ae-4603-4465-8844-6d905bc97c22 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.310501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.311528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.311815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.312092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] PATH_INFO: `/v3/projects/943a034c75564ffd95ccf2f6c27457e7/users/15d09ed5c1fc4b118e5533278d5b9524/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.316939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100623a8bec74c699e7f41561b92b73b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:00.317723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100623a8bec74c699e7f41561b92b73b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:00.325468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100623a8bec74c699e7f41561b92b73b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:00.327863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=943a034c75564ffd95ccf2f6c27457e7, user_id=15d09ed5c1fc4b118e5533278d5b9524, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.330716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e882d-5d2c-48d8-9cd3-894c810dbd7f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.333591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100623a8bec74c699e7f41561b92b73b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:00.334527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bee17a57-9b33-4b32-aa75-1156bd470c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100623a8bec74c699e7f41561b92b73b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:00.345907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1102/2186] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:00 2022] PUT /identity/v3/projects/943a034c75564ffd95ccf2f6c27457e7/users/15d09ed5c1fc4b118e5533278d5b9524/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:00.357425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.357730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.357980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.361028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1085/2187] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:00 2022] DELETE /identity/v3/users/100623a8bec74c699e7f41561b92b73b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:00.366583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.366900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.367211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.393955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:00.395338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.397091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] MFA Rules not processed for user `15d09ed5c1fc4b118e5533278d5b9524`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:00.399063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.488888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1086/2188] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:00.498875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.502689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1103/2189] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:00.503675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.504628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.505607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:00.521694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e313-5c8b-408d-925d-01373f442ea4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.530315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:00.530521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.531183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.531495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.531813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.532470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.534580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.540408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:00.583925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1104/2190] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:08:00.587322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.621910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c798f-42c8-412f-8bde-3ca52c3f9114 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.679740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956188-0fe9-4ce1-8c83-dd2dd4ac54c3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.688247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.689135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.689715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.690078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.690956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.693732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.713992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.715002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.715490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.715931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.730787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.733751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.734295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1105/2191] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:00.738747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.748536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1087/2192] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:00.749746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d5db-035f-43d6-8c04-79a2a365b312 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.755951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.756634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.757129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.757487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.758605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.760204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.775295 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1106/2193] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:00.777633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.790384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62b02-cee1-45cd-b092-446c697a7540 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.795374 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.796215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.796468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.796743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] PATH_INFO: `/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf/users/7d71f03010d44ad585609b449f3f80e0/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.814515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652ddccdcdd3481fbe5ddaa19b1688cf, user_id=7d71f03010d44ad585609b449f3f80e0, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.817404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.831534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1088/2194] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:00 2022] PUT /identity/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf/users/7d71f03010d44ad585609b449f3f80e0/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:00.835723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.847231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0448b99-718e-4576-a1b0-258981e4b06f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.907171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.907787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.908061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.908354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.908997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.910599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c08d19-6126-4326-8b69-0fb455fca555 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.927874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1107/2195] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:00.928947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:00.940406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9baab-5f72-4517-b7e9-cd10c4804435 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:00.945816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:00.946496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.946769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.947063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] PATH_INFO: `/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf/users/7d71f03010d44ad585609b449f3f80e0/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:00.949024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652ddccdcdd3481fbe5ddaa19b1688cf, user_id=7d71f03010d44ad585609b449f3f80e0, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:00.951827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:00.963283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1089/2196] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:00 2022] PUT /identity/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf/users/7d71f03010d44ad585609b449f3f80e0/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:00.974223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:00.974514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:00.974792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.011085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] MFA Rules not processed for user `7d71f03010d44ad585609b449f3f80e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:01.012339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.055847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.057883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.067934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:01.099981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1108/2197] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:01.112804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.114616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.130493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5d56-c12c-4627-885c-43854dbb92e1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.135516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.136214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.136568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.136843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] PATH_INFO: `/v3/projects/e8da30a9a7b34fc6a0baa6021ff4eccc` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.137978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8da30a9a7b34fc6a0baa6021ff4eccc)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.140185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.176759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90aa63-72c0-4b8f-bb8e-8b0603fa42f4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.186550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:01.190581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8da30a9a7b34fc6a0baa6021ff4eccc', 'request_id': 'req-1d7642d3-2699-4b6c-956b-42460dc73236'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.197687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8da30a9a7b34fc6a0baa6021ff4eccc', 'request_id': 'req-1d7642d3-2699-4b6c-956b-42460dc73236'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.203568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8da30a9a7b34fc6a0baa6021ff4eccc', 'request_id': 'req-1d7642d3-2699-4b6c-956b-42460dc73236'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.217726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] The token cache is being invalidate because project e8da30a9a7b34fc6a0baa6021ff4eccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:01.218013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.218642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1d7642d3-2699-4b6c-956b-42460dc73236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.220203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1090/2198] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:01 2022] DELETE /identity/v3/projects/e8da30a9a7b34fc6a0baa6021ff4eccc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:01.224937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.225441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.225564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.233040 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.233762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.234039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.234317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.236279 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.245770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:01.246953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.301163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.302615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df2ea8-2ad9-4272-98a6-f57e50fca1d3 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.312072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1109/2199] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:01.314738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1091/2200] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:01.322483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.322770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.323052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.344101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:01.345333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.353790 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1110/2201] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:01.358401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.368846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ca2d-9a99-4dd8-9852-584fe68becff None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.373818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.374513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.374788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.375134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.375913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:01.376315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.378425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.391527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1092/2202] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:08:01.396008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.406133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc9a0a-167b-4305-972f-72d19ac3f402 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.410980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.411610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.411889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.412160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.412824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.414728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.446883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1111/2203] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:01.447740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.458837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3204f-f553-4455-a41c-918396c6cd0c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.523976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.523976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.523976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.523976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.524284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.525573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.552355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1093/2204] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:01.553390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.564487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b5732-a99d-4df9-9cc0-90a422cd1d5b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.569211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.569994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.570140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.570477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.571093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.574906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.587323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1112/2205] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:01.588699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.599490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4394f2f-3669-43ea-bbc0-2a5e83326d9f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.604476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.605081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.605362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.605632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.606274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.608247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae94f5-ac73-4f2a-a014-c6a349e94aa4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.619978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1094/2206] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:01.624211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.635503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4536b-443d-4ff5-8f32-f998f9f4e985 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.640969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.641748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.642039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.642315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] PATH_INFO: `/v3/projects/22dd369a5e994d92af360b615777e098/users/79403e726f2e4bf5805c1c024832b392/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.660045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:01.665061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dd369a5e994d92af360b615777e098, user_id=79403e726f2e4bf5805c1c024832b392, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.668333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.688187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1113/2207] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:01 2022] PUT /identity/v3/projects/22dd369a5e994d92af360b615777e098/users/79403e726f2e4bf5805c1c024832b392/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:01.698488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.698787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.699113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.722803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.724673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.730614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] MFA Rules not processed for user `79403e726f2e4bf5805c1c024832b392`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:01.731832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.758777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:01.793664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f3ca-b83e-44cd-98df-9f26886bed1b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.800795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.802610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.803135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.803316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] PATH_INFO: `/v3/users/15d09ed5c1fc4b118e5533278d5b9524` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.804992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d09ed5c1fc4b118e5533278d5b9524)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.805747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1095/2208] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:01.807258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.810241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.821762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adfbfc-042a-4faf-8bbc-b2067d1d8478 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.827912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:01.828629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.829010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.829381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.830118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:01.832159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:01.861116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d09ed5c1fc4b118e5533278d5b9524'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.867186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1096/2209] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:08:01.870137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:01.875493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d09ed5c1fc4b118e5533278d5b9524'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.883094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d09ed5c1fc4b118e5533278d5b9524'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.883900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d09ed5c1fc4b118e5533278d5b9524'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.884591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-409e8a6b-20aa-4d76-bf53-b953612061d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d09ed5c1fc4b118e5533278d5b9524'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:01.899785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1114/2210] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:01 2022] DELETE /identity/v3/users/15d09ed5c1fc4b118e5533278d5b9524 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:01.904145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:01.904724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:01.904836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:01.925448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:01.925841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:01.947183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ca7d5-6e51-4473-a27d-d91663e6b726 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:01.997728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1115/2211] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:02.007524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.008901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.018230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.018865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.019172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.019570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.019860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:02.020208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.022130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.048092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1097/2212] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:01 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:08:02.052201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.053864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.067527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70773-45d6-4bd3-907f-86828b7b66b2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.070212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea4a8a-2cdd-4ce3-9126-1de346403ff1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.072547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.073141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.073419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.073686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.074328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.074441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.074937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.075315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.075586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.076196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.086042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.090601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.090717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1098/2213] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:02.092318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.097799 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1116/2214] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:02.103644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6aa25-39b7-4734-9fe9-c22b11263bf0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.110356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.111052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.111328 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.111602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] PATH_INFO: `/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931/users/8d66d772d15e4f95a713ca2b29c59458/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.129105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7af2f0a254bcb9ece0a4bfcd34931, user_id=8d66d772d15e4f95a713ca2b29c59458, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.131897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.145366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1099/2215] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:02 2022] PUT /identity/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931/users/8d66d772d15e4f95a713ca2b29c59458/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:02.149443 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.159830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692cf82-90de-49b0-b3ee-354ae1e6e31d service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.218312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.218916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.219236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.219495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.220128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.221700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff76426-5ea6-4239-840d-24a3a34113f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.231773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1117/2216] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:02.236066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.247003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cf29-81cc-4cc9-8458-74e44df12493 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.252172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.252828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.253108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.253420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] PATH_INFO: `/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931/users/8d66d772d15e4f95a713ca2b29c59458/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.255396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7af2f0a254bcb9ece0a4bfcd34931, user_id=8d66d772d15e4f95a713ca2b29c59458, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.258085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.260475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:02.275559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1100/2217] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:02 2022] PUT /identity/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931/users/8d66d772d15e4f95a713ca2b29c59458/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:02.282309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.282669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.282995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.315978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] MFA Rules not processed for user `8d66d772d15e4f95a713ca2b29c59458`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:02.317366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.392406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1118/2218] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:02.399215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.401206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.404755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.406217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.453498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:02.471724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68eb07-ae35-4b59-96fb-6e7a2256b409 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.476012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36994-9497-4be2-9f7f-fca11d708ccb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.476488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.477122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.477427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.477699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] PATH_INFO: `/v3/projects/943a034c75564ffd95ccf2f6c27457e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.478811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=943a034c75564ffd95ccf2f6c27457e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.480130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.480761 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.480947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.481228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.481525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.483424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.527869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943a034c75564ffd95ccf2f6c27457e7', 'request_id': 'req-9234b0fb-efb2-403c-a715-bcd791c93426'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.534171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943a034c75564ffd95ccf2f6c27457e7', 'request_id': 'req-9234b0fb-efb2-403c-a715-bcd791c93426'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.539928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943a034c75564ffd95ccf2f6c27457e7', 'request_id': 'req-9234b0fb-efb2-403c-a715-bcd791c93426'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.544896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.546979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.553003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] The token cache is being invalidate because project 943a034c75564ffd95ccf2f6c27457e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:02.553498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.554333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.555923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1101/2219] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:02 2022] DELETE /identity/v3/projects/943a034c75564ffd95ccf2f6c27457e7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:02.556917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1119/2220] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:02.559898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:02.561817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:02.626222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234b0fb-efb2-403c-a715-bcd791c93426 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:02.631011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:02.631663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:02.631949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:02.632243 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] PATH_INFO: `/v3/users/7d71f03010d44ad585609b449f3f80e0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:02.633487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d71f03010d44ad585609b449f3f80e0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:02.635462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:02.666311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:02.688440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d71f03010d44ad585609b449f3f80e0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.689551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d71f03010d44ad585609b449f3f80e0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.696871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d71f03010d44ad585609b449f3f80e0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.711795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d71f03010d44ad585609b449f3f80e0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.712456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4436fbb6-3461-432e-b02a-52ba90329723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d71f03010d44ad585609b449f3f80e0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:02.735464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1102/2221] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:02 2022] DELETE /identity/v3/users/7d71f03010d44ad585609b449f3f80e0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:03.156386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.170738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4d385-b6d0-4f61-8721-30b8c9a7b8cc service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.249645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.250319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.250592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.250860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] PATH_INFO: `/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.252080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652ddccdcdd3481fbe5ddaa19b1688cf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.253966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.312950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652ddccdcdd3481fbe5ddaa19b1688cf', 'request_id': 'req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:03.319654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652ddccdcdd3481fbe5ddaa19b1688cf', 'request_id': 'req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:03.326256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652ddccdcdd3481fbe5ddaa19b1688cf', 'request_id': 'req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:03.343000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] The token cache is being invalidate because project 652ddccdcdd3481fbe5ddaa19b1688cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:03.343326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:03.344396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3d324833-0753-4cdb-9d7a-02c39bbd3ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:03.345961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1120/2222] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:03 2022] DELETE /identity/v3/projects/652ddccdcdd3481fbe5ddaa19b1688cf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:03.352594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.352594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.352594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.376792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:03.378005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:03.466862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1103/2223] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:03.477518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.477866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.478275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.508271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:03.509499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:03.523114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1121/2224] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:03.529389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.540388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e58eaa-f19a-4d06-8036-b9322d825cea None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.547583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.548203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.548481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.548903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.550216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:03.550717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.553565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.570156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1104/2225] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:08:03.575001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.590294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf484c0-dceb-4fc0-b010-303eec08cdbe None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.596214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.596809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.597092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.597353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.597985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.599977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.610566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:03.641294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1122/2226] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:08:03.646339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.657595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fe837-4146-49a5-823b-5bcde64eeb0a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.740608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.741656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.742164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.742589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.743795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.746983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.771654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1105/2227] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:03.777115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.787902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3f3ee-a9d6-43cf-9dce-01b4b5b2910e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.793265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.793865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.794173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.794722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.795292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.797051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f31e77-36fb-46fe-8632-7d7d126da57e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.817537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1123/2228] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:03.821618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.833111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6136d1-14d3-4d9c-8159-89c1ef04bbd8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.838828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.839478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.839733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.840000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.840641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.842261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.855845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.856144 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.856421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.857624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1106/2229] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:03.861826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:03.873570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f394e7-151c-4f44-afe0-8248d29a19cc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:03.876434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] MFA Rules not processed for user `3fa26512830547db839cea342721da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:03.877683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:03.878771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755778af-9c32-4084-a452-df18d528278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:03.879434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755778af-9c32-4084-a452-df18d528278b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.879712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755778af-9c32-4084-a452-df18d528278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.879983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755778af-9c32-4084-a452-df18d528278b admin admin] PATH_INFO: `/v3/projects/6053fe28c832470393d4a6f593fc75fb/users/89cb71e751d8405b877adc12e53d34a7/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.883693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:03.900070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755778af-9c32-4084-a452-df18d528278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6053fe28c832470393d4a6f593fc75fb, user_id=89cb71e751d8405b877adc12e53d34a7, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:03.903196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755778af-9c32-4084-a452-df18d528278b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:03.921414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1107/2230] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:03 2022] PUT /identity/v3/projects/6053fe28c832470393d4a6f593fc75fb/users/89cb71e751d8405b877adc12e53d34a7/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:03.931493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:03.931770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:03.932037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:03.941997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1124/2231] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:03.971369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] MFA Rules not processed for user `89cb71e751d8405b877adc12e53d34a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:03.972998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:04.075529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1108/2232] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:04.077021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.098994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32da8e-3003-4b92-8261-99146f1a100d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.185109 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.186084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.186620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.186989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.187762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.189931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.245971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1125/2233] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:04 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:04.250779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.264156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6688-30bc-4919-b3e5-0a97aeb83d68 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.336087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.336705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.336990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.337261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.337899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.339966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.368948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1109/2234] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:04 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:04.370370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.381988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9351f1-2e1e-4fbb-9768-012acbc15541 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.386939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.387716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.387995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.388372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.389260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.391059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.405930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1126/2235] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:04.408068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.419571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae1a0-1cfc-48b1-a1fd-b9f62d5f356f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.424838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.425458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.425904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.426181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.426855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.428595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0816b-639e-4660-af5b-99146a51baa7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.444120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1110/2236] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:04.445487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.457388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce115a-f1a3-4470-bc80-24c1c920e88f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.463491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.464233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.464513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.464783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] PATH_INFO: `/v3/projects/bc8a774cac904272b6cb4e9c76ed3e6f/users/96bef99354614663b2b8237ae6a719f4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.483707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a774cac904272b6cb4e9c76ed3e6f, user_id=96bef99354614663b2b8237ae6a719f4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.486408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.500015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1127/2237] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:04 2022] PUT /identity/v3/projects/bc8a774cac904272b6cb4e9c76ed3e6f/users/96bef99354614663b2b8237ae6a719f4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:04.511885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.512478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.512478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.548795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] MFA Rules not processed for user `96bef99354614663b2b8237ae6a719f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:04.548795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:04.594163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:04.622100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1111/2238] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:04.626644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.661358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:04.695310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134c98-7001-47cf-8491-1525d1fd5b47 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.708289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.709332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.709804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.710238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.711044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.713320 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.746266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1128/2239] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:04 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:04.751292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.763859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207acbe-29a7-4251-abe0-5b260bb4184b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.845876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.846560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.846805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.847102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.847750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.849915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.879737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1112/2240] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:04 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:04.881431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.899559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42ef15-60cb-4759-b941-996e88c9f3b6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.904922 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.905531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.905840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.906057 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.906859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.908575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.921011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1129/2241] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:04.925721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.936614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8bb00-143f-494d-af1c-6eeb21aa6d4f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:04.941769 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:04.942428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:04.942710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:04.942987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] PATH_INFO: `/v3/projects/dce203aba16c4568a9e7036a2b76fd10/users/49c1476521a24ae28fb231718283ecb6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:04.960216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce203aba16c4568a9e7036a2b76fd10, user_id=49c1476521a24ae28fb231718283ecb6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:04.963261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:04.978757 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1113/2242] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:04 2022] PUT /identity/v3/projects/dce203aba16c4568a9e7036a2b76fd10/users/49c1476521a24ae28fb231718283ecb6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:04.983175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:04.993633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df771c-3be3-40d6-8484-2e719685d29a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.056566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.057235 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.057524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.057842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.058523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.060349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483e9ad-efec-48bc-aac0-82df12fd5e63 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.071712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1130/2243] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:05.076185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:05.087537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc805c-15ae-48d4-b6f3-dde2374d499a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.093116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.093880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.094431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.094655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] PATH_INFO: `/v3/projects/dce203aba16c4568a9e7036a2b76fd10/users/49c1476521a24ae28fb231718283ecb6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.096950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce203aba16c4568a9e7036a2b76fd10, user_id=49c1476521a24ae28fb231718283ecb6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.099777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.113134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1114/2244] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:05 2022] PUT /identity/v3/projects/dce203aba16c4568a9e7036a2b76fd10/users/49c1476521a24ae28fb231718283ecb6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:05.123899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.124198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.124767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.155389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] MFA Rules not processed for user `49c1476521a24ae28fb231718283ecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:05.156632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.211363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:05.213448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.245523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1131/2245] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:05.270489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:05.272100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.341215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:05.347062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dcbd5-30b2-4e03-af30-b6564354d04b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.358433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.359120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.359398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.359670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] PATH_INFO: `/v3/users/79403e726f2e4bf5805c1c024832b392` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.361074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79403e726f2e4bf5805c1c024832b392)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.363861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.381254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf051b-ca4b-44a9-8ae7-36ec4eb551c2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.385426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.386199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.386840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.387437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.400706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.402736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b0ba2-68e7-477a-a03e-9e443d3f76a1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.412569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:05.414993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1132/2246] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:05.426240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79403e726f2e4bf5805c1c024832b392'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:05.427425 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79403e726f2e4bf5805c1c024832b392'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:05.438164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79403e726f2e4bf5805c1c024832b392'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:05.448134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79403e726f2e4bf5805c1c024832b392'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:05.448864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79403e726f2e4bf5805c1c024832b392'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:05.470787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1115/2247] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:05 2022] DELETE /identity/v3/users/79403e726f2e4bf5805c1c024832b392 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:05.475499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.475791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.476380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.484002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:05.485816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.497121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:05.498477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.547550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243a3df-bbf2-4307-a353-cd593cb06690 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.553357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.554190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.554535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.555016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.557183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.566874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1133/2248] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:05.577380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:05.579615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:05.649323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.651015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.658632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e70e51-d48c-44d1-a412-7fef2c1639de None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:05.661119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1116/2249] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:05.662626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:05.663817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:05.663817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:05.663981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:05.674505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:05.676662 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:05.689865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1134/2250] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:06.084360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:06.085786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:06.108531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2d84-be1e-4374-b93f-7e3e3b15928f service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:06.119966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:06.120694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.121039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.121377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] PATH_INFO: `/v3/projects/22dd369a5e994d92af360b615777e098` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.122669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dd369a5e994d92af360b615777e098)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:06.125311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:06.177446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dd369a5e994d92af360b615777e098', 'request_id': 'req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.183412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dd369a5e994d92af360b615777e098', 'request_id': 'req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.188950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dd369a5e994d92af360b615777e098', 'request_id': 'req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.190605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:06.204815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] The token cache is being invalidate because project 22dd369a5e994d92af360b615777e098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:06.205275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.206086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.208574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1117/2251] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:06 2022] DELETE /identity/v3/projects/22dd369a5e994d92af360b615777e098 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:06.212981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:06.223331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991943a-40d4-462b-8e40-e23f0a9abe8d service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:06.284654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:06.285308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.285554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.285803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] PATH_INFO: `/v3/users/8d66d772d15e4f95a713ca2b29c59458` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.286935 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d66d772d15e4f95a713ca2b29c59458)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:06.289076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:06.305917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:06.349140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d66d772d15e4f95a713ca2b29c59458'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.378566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d66d772d15e4f95a713ca2b29c59458'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.386757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d66d772d15e4f95a713ca2b29c59458'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.387659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d66d772d15e4f95a713ca2b29c59458'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.388410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aed097e1-1f59-41d7-9d21-2907f1d41314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d66d772d15e4f95a713ca2b29c59458'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.407064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1135/2252] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:06 2022] DELETE /identity/v3/users/8d66d772d15e4f95a713ca2b29c59458 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:06.489404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:06.491105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:06.506649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6ff5-bbbc-4aa6-9821-bb56aaaa190a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:06.599198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:06.600221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.600608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.603875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] PATH_INFO: `/v3/users/9763c5457b444ab0a2486f183d25c908` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.606746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9763c5457b444ab0a2486f183d25c908)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:06.614049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:06.673626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9763c5457b444ab0a2486f183d25c908'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.674198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9763c5457b444ab0a2486f183d25c908'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.681866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9763c5457b444ab0a2486f183d25c908'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.694111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9763c5457b444ab0a2486f183d25c908'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.694776 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9763c5457b444ab0a2486f183d25c908'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.720318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1118/2253] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:06 2022] DELETE /identity/v3/users/9763c5457b444ab0a2486f183d25c908 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:06.722359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.722759 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.722970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.744523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:06.745975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:06.758926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:06.761212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:06.844039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1136/2254] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:06.855001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:06.856492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:06.878750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437bf7-aebd-43a1-af73-7850a02760b2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:06.885074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:06.885768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.886045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.886330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] PATH_INFO: `/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.887572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f7af2f0a254bcb9ece0a4bfcd34931)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:06.890092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:06.906680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:06.936488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b97398-1de4-499b-8d77-3c58e3390290 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:06.945642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:06.946289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:06.946585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:06.946906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:06.958260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:06.959748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1b9ab-c6f1-40be-bca7-000247b6bb2f service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:06.971519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1137/2255] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:06.971944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f7af2f0a254bcb9ece0a4bfcd34931', 'request_id': 'req-d28a1c7f-cf23-4425-92c7-67a7aa466d08'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.980068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f7af2f0a254bcb9ece0a4bfcd34931', 'request_id': 'req-d28a1c7f-cf23-4425-92c7-67a7aa466d08'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:06.990133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f7af2f0a254bcb9ece0a4bfcd34931', 'request_id': 'req-d28a1c7f-cf23-4425-92c7-67a7aa466d08'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.003506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] The token cache is being invalidate because project 19f7af2f0a254bcb9ece0a4bfcd34931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:07.003791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.004439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d28a1c7f-cf23-4425-92c7-67a7aa466d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.007637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1119/2256] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:06 2022] DELETE /identity/v3/projects/19f7af2f0a254bcb9ece0a4bfcd34931 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:07.011390 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.012030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.012463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.042030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:07.044555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:07.128293 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:07.175408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1138/2257] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:07.177177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.177793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.179171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.201596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:07.203099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:07.215259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1120/2258] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:07.225513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.240226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee5d5-6367-4ea1-9b0e-1ed13dd79d63 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.251421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.252377 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.254853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.255422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.256444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:07.263345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.264926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.281473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1139/2259] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:08:07.290156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.306260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25ab11-1984-435c-b6e1-7077964a8059 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.311237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.311848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.312120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.312382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.313283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.320652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.365803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1121/2260] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:07 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:07.367312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.378838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183bb8-f0ae-412b-a546-246836d08f5c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.405416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.407332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:07.609816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.610450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.612356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.612632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.613593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.620209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.636981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa4f7e-3b01-4e09-8291-02d86d9250b1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.651437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.652177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.652478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.652740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] PATH_INFO: `/v3/projects/92e55122214e439f897e2bafd1718c44` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.656020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e55122214e439f897e2bafd1718c44)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.662100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.668967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1140/2261] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:07 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:07.679208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.696200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7e318-d364-4932-813c-301b14905c91 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.710586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.711274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.711576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.711851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.712524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.714201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.735017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1141/2262] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:07.741434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.761123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac671ce5-5bb5-4407-9c45-bac1e4a99ac8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.768853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.769802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.770190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.770640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.771803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.773631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:07.793486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1142/2263] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:07.797709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:07.817274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa86d-37ee-4ba7-a683-14e862442613 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:07.831297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e55122214e439f897e2bafd1718c44', 'request_id': 'req-290dabe6-4425-4478-85bd-daabcbaf046b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.854338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e55122214e439f897e2bafd1718c44', 'request_id': 'req-290dabe6-4425-4478-85bd-daabcbaf046b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.872778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e55122214e439f897e2bafd1718c44', 'request_id': 'req-290dabe6-4425-4478-85bd-daabcbaf046b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.897115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] The token cache is being invalidate because project 92e55122214e439f897e2bafd1718c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:07.897442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.898748 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-290dabe6-4425-4478-85bd-daabcbaf046b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:07.901409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1122/2264] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:07 2022] DELETE /identity/v3/projects/92e55122214e439f897e2bafd1718c44 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:07.964552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:07.966255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:07.966691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:07.967163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] PATH_INFO: `/v3/projects/6e93e81f9bb0432f8feaec2d8da85a47/users/139032a1d6d54ddd9981db349c38bbfe/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:07.988453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e93e81f9bb0432f8feaec2d8da85a47, user_id=139032a1d6d54ddd9981db349c38bbfe, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:07.991825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:08.013570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1143/2265] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:07 2022] PUT /identity/v3/projects/6e93e81f9bb0432f8feaec2d8da85a47/users/139032a1d6d54ddd9981db349c38bbfe/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:08.030558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:08.030859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:08.039403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:08.073734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:08.152223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] MFA Rules not processed for user `139032a1d6d54ddd9981db349c38bbfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:08.153717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:08.229466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1123/2266] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:08.241769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:08.243518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:08.276018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:08.320052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb3f8b-06d3-4439-9c4a-b6d1c0bae368 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:08.323897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:08.324581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:08.324927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:08.325286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:08.334458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:08.335927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72a42a-f1f3-4c9c-9f78-7d6c9a6fb8b1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:08.346540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1144/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:09.019042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:09.020562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:09.101509 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720aa223-10c9-4d9e-bda5-c21161c547f0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:09.106155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:09.106891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:09.107436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:09.107755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] PATH_INFO: `/v3/users/89cb71e751d8405b877adc12e53d34a7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:09.108991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=89cb71e751d8405b877adc12e53d34a7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:09.111281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:09.163664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89cb71e751d8405b877adc12e53d34a7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.164326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89cb71e751d8405b877adc12e53d34a7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.172680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89cb71e751d8405b877adc12e53d34a7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.182767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89cb71e751d8405b877adc12e53d34a7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.183423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89cb71e751d8405b877adc12e53d34a7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.202304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1124/2268] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:09 2022] DELETE /identity/v3/users/89cb71e751d8405b877adc12e53d34a7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:09.210722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:09.211328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:09.211618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:09.233252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:09.234503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:09.303414 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1145/2269] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:09.313500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:09.315057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:09.329788 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:09.381130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1db4-b05a-43fd-86ec-a5b699f44bfc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:09.385407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:09.386034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:09.386316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:09.386584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:09.396582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:09.398020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:09.408130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1125/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:09.716348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:09.717845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:09.732837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036dc47-6c1e-4b98-ae10-8112acdd84b0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:09.738807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:09.739550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:09.739890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:09.740160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] PATH_INFO: `/v3/projects/6053fe28c832470393d4a6f593fc75fb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:09.741984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6053fe28c832470393d4a6f593fc75fb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:09.744203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:09.810563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6053fe28c832470393d4a6f593fc75fb', 'request_id': 'req-b718d31f-bd17-42c8-bd47-98e5386fb068'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.818623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6053fe28c832470393d4a6f593fc75fb', 'request_id': 'req-b718d31f-bd17-42c8-bd47-98e5386fb068'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.825197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6053fe28c832470393d4a6f593fc75fb', 'request_id': 'req-b718d31f-bd17-42c8-bd47-98e5386fb068'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.839255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] The token cache is being invalidate because project 6053fe28c832470393d4a6f593fc75fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:09.839399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.840100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:09.841876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1146/2271] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:09 2022] DELETE /identity/v3/projects/6053fe28c832470393d4a6f593fc75fb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:09.845882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:09.856942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f9a01-302f-4cb9-9fa9-83b4c7d8a4fd service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:09.928589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:09.929308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:09.929773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:09.930181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] PATH_INFO: `/v3/users/96bef99354614663b2b8237ae6a719f4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:09.931422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96bef99354614663b2b8237ae6a719f4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:09.933436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:10.009567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96bef99354614663b2b8237ae6a719f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.010228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96bef99354614663b2b8237ae6a719f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.028719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96bef99354614663b2b8237ae6a719f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.037905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96bef99354614663b2b8237ae6a719f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.038652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96bef99354614663b2b8237ae6a719f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.057772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1126/2272] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:09 2022] DELETE /identity/v3/users/96bef99354614663b2b8237ae6a719f4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:10.359385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:10.360908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:10.376090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d31f-bd17-42c8-bd47-98e5386fb068 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:10.419447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:10.437652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:10.438344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:10.438627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:10.438903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] PATH_INFO: `/v3/projects/bc8a774cac904272b6cb4e9c76ed3e6f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:10.440231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8a774cac904272b6cb4e9c76ed3e6f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:10.442575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:10.498660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8a774cac904272b6cb4e9c76ed3e6f', 'request_id': 'req-e7d32e91-613b-45e0-ac06-c6602b30d354'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.505354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8a774cac904272b6cb4e9c76ed3e6f', 'request_id': 'req-e7d32e91-613b-45e0-ac06-c6602b30d354'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.511266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8a774cac904272b6cb4e9c76ed3e6f', 'request_id': 'req-e7d32e91-613b-45e0-ac06-c6602b30d354'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.525047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] The token cache is being invalidate because project bc8a774cac904272b6cb4e9c76ed3e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:10.525332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.525949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.527497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1147/2273] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:10 2022] DELETE /identity/v3/projects/bc8a774cac904272b6cb4e9c76ed3e6f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:10.531770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:10.533161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:10.546427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c343c2-89ba-490f-9b6f-fc07c5871620 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:10.582494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:10.614121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:10.614752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:10.615107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:10.615394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] PATH_INFO: `/v3/users/49c1476521a24ae28fb231718283ecb6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:10.616601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c1476521a24ae28fb231718283ecb6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:10.618550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:10.668987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c1476521a24ae28fb231718283ecb6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.670045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c1476521a24ae28fb231718283ecb6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.677794 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c1476521a24ae28fb231718283ecb6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.686472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c1476521a24ae28fb231718283ecb6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.687138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-148649a3-c4dc-4a46-91a6-d975d79c57bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c1476521a24ae28fb231718283ecb6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:10.709088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1127/2274] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:10 2022] DELETE /identity/v3/users/49c1476521a24ae28fb231718283ecb6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:11.011131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.012605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:11.028980 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d32e91-613b-45e0-ac06-c6602b30d354 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.096663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.097304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.097580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.097852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] PATH_INFO: `/v3/projects/dce203aba16c4568a9e7036a2b76fd10` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.098958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce203aba16c4568a9e7036a2b76fd10)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.101043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.153254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce203aba16c4568a9e7036a2b76fd10', 'request_id': 'req-17d37f67-7bd3-49d7-9e86-625e90b1d878'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:11.160263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce203aba16c4568a9e7036a2b76fd10', 'request_id': 'req-17d37f67-7bd3-49d7-9e86-625e90b1d878'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:11.166883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce203aba16c4568a9e7036a2b76fd10', 'request_id': 'req-17d37f67-7bd3-49d7-9e86-625e90b1d878'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:11.183426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] The token cache is being invalidate because project dce203aba16c4568a9e7036a2b76fd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:11.183861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:11.184549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-17d37f67-7bd3-49d7-9e86-625e90b1d878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:11.186569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1148/2275] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:11 2022] DELETE /identity/v3/projects/dce203aba16c4568a9e7036a2b76fd10 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:11.192446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.192936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.193028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.214113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:11.215355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:11.295534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1128/2276] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:11.303937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.304217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.304491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.312745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:11.328553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:11.329952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:11.338537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1149/2277] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:11.343536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.354681 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034d964-def9-4afd-91a0-9f9daee4bb8e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.361049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.361683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.362030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.362246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.363424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:11.363755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.365424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.379262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1129/2278] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:08:11.383990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.394336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142cea1-1719-4733-bc66-d62855ecc214 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.399123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.399762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.400091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.400362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.401071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.402977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.432016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1150/2279] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:11.436402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.446806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dba83-2aaf-47ab-9339-1f35db0cba3a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.490262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:11.511957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.512565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.512838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.513321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.514060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.516033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.541293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1130/2280] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:11.542559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.554431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66383-2b72-4647-8c60-afe59f7538b5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.560838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.561461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.561722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.561989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.562813 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.564508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.575092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1151/2281] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:11.579753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.590584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe052f-df12-4fa5-b3ad-9bd81f06ee02 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.596703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.597312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.597567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.597958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.598636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.600312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77f720-334e-40e3-8479-2fef2fe657bf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.615654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1131/2282] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:11.616837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.627124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6a99e-0ee2-4f46-8c35-2004098c9da2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.632103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.632749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.633019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.633291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] PATH_INFO: `/v3/projects/e6c414815a1d477e99ad5e5932d180c5/users/acb74fd856104ea087691c8a901f9248/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.648374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c414815a1d477e99ad5e5932d180c5, user_id=acb74fd856104ea087691c8a901f9248, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.651171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.667539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1152/2283] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:11 2022] PUT /identity/v3/projects/e6c414815a1d477e99ad5e5932d180c5/users/acb74fd856104ea087691c8a901f9248/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:11.680557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.680842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.681115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.713582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] MFA Rules not processed for user `acb74fd856104ea087691c8a901f9248`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:11.714803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:11.787848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1132/2284] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:11.793176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.861964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed21cd-5284-467a-9267-a0f712b77c9e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.867701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.868254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.868534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.868801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.869408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.871399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:11.901948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1153/2285] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:11.906381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:11.918012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa249a-da13-42e3-bb53-ff55ba474bf0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:11.980069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:11.980691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:11.980977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:11.981231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:11.981953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:11.984034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.009400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1133/2286] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:11 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:12.013996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:12.024855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85573b7e-ba77-4f07-827e-005cec523161 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:12.033817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:12.034449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.034726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.034994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.035671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:12.037307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.053184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1154/2287] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:08:12.054376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:12.064707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d81c6-ce8d-49f5-8dcd-4c966f8ef0cf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:12.069637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:12.070291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.070571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.070842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] PATH_INFO: `/v3/projects/390c449deb4c4feb992108f500d4f8f4/users/ab58af37d2d4444eb66daa0b7d139109/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.087905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=390c449deb4c4feb992108f500d4f8f4, user_id=ab58af37d2d4444eb66daa0b7d139109, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:12.090741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.108866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1134/2288] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:12 2022] PUT /identity/v3/projects/390c449deb4c4feb992108f500d4f8f4/users/ab58af37d2d4444eb66daa0b7d139109/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:12.112764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:12.124629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591510b5-8c22-4d0f-b55e-f6e86ebc6044 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:12.183699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:12.184342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.184627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.184925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.185962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:12.187608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303c146-84d1-4f87-8806-62de7eaa2156 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.198188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1155/2289] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:12.203060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:12.214438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470e4f6-faa1-4e12-b4d9-2519f03d8f60 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:12.220815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:12.221491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.221771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.222036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] PATH_INFO: `/v3/projects/390c449deb4c4feb992108f500d4f8f4/users/ab58af37d2d4444eb66daa0b7d139109/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.224367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] RBAC: Authorizing `identity:create_grant(project_id=390c449deb4c4feb992108f500d4f8f4, user_id=ab58af37d2d4444eb66daa0b7d139109, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:12.227153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.235144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:12.254455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1135/2290] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:12 2022] PUT /identity/v3/projects/390c449deb4c4feb992108f500d4f8f4/users/ab58af37d2d4444eb66daa0b7d139109/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:12.268771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.269297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.269927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.298142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:12.310975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] MFA Rules not processed for user `ab58af37d2d4444eb66daa0b7d139109`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:12.313009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:12.389220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1156/2291] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:12.400689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:12.402512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:12.468931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d319e-9eb6-4a93-841d-42b5e31c955d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:12.473455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:12.474061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:12.474337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:12.474605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:12.484215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:12.485660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf60b66-12e8-4c85-b1b7-a14ae5fb599a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:12.495620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1136/2292] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:13.103874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:13.180195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a79ecb-7cd4-461f-aa3e-bc51e62f31b2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:13.184949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:13.185637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:13.185915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:13.186167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] PATH_INFO: `/v3/users/acb74fd856104ea087691c8a901f9248` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:13.187335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb74fd856104ea087691c8a901f9248)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:13.189345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:13.243600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb74fd856104ea087691c8a901f9248'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.256505 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb74fd856104ea087691c8a901f9248'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.265248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb74fd856104ea087691c8a901f9248'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.266056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb74fd856104ea087691c8a901f9248'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.266712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb74fd856104ea087691c8a901f9248'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.284823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1157/2293] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:13 2022] DELETE /identity/v3/users/acb74fd856104ea087691c8a901f9248 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:13.289520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:13.290277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:13.290802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:13.314506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:13.315808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:13.376680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1137/2294] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:13.386867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:13.388719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:13.407239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:13.455654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec5833-9ae3-414e-bdfe-5459ee99ce3a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:13.459338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:13.459944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:13.460221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:13.460486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:13.471998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:13.473436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:13.483172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1158/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:13.817278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:13.819376 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:13.841304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd49b-d788-4ed6-a8a9-b7c4400fb1e7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:13.846888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:13.847550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:13.848094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:13.848405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] PATH_INFO: `/v3/projects/e6c414815a1d477e99ad5e5932d180c5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:13.849488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c414815a1d477e99ad5e5932d180c5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:13.851426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:13.898976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c414815a1d477e99ad5e5932d180c5', 'request_id': 'req-c919e742-891c-49e3-ad57-08590c2723d7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.905347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c414815a1d477e99ad5e5932d180c5', 'request_id': 'req-c919e742-891c-49e3-ad57-08590c2723d7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.910871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c414815a1d477e99ad5e5932d180c5', 'request_id': 'req-c919e742-891c-49e3-ad57-08590c2723d7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.924497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] The token cache is being invalidate because project e6c414815a1d477e99ad5e5932d180c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:13.924785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.925448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:13.926979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1138/2296] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:13 2022] DELETE /identity/v3/projects/e6c414815a1d477e99ad5e5932d180c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:13.931147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:13.940908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073007fd-3f99-4c9d-9e1b-5ad761c5f510 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:14.002852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:14.003530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.003811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.004088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] PATH_INFO: `/v3/users/ab58af37d2d4444eb66daa0b7d139109` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.005229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab58af37d2d4444eb66daa0b7d139109)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:14.007305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:14.056432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab58af37d2d4444eb66daa0b7d139109'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.068421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab58af37d2d4444eb66daa0b7d139109'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.075776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab58af37d2d4444eb66daa0b7d139109'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.076604 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab58af37d2d4444eb66daa0b7d139109'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.077255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-20173b02-1e38-4c0f-b5be-30c740a3bd74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab58af37d2d4444eb66daa0b7d139109'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.090856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1159/2297] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:13 2022] DELETE /identity/v3/users/ab58af37d2d4444eb66daa0b7d139109 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:14.543541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:14.545379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:14.558827 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919e742-891c-49e3-ad57-08590c2723d7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:14.587227 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:14.640290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:14.641104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.641499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.641893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] PATH_INFO: `/v3/projects/390c449deb4c4feb992108f500d4f8f4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.643162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=390c449deb4c4feb992108f500d4f8f4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:14.645380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:14.698578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390c449deb4c4feb992108f500d4f8f4', 'request_id': 'req-172205d9-408d-4335-a430-e7b8e1210981'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.706132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390c449deb4c4feb992108f500d4f8f4', 'request_id': 'req-172205d9-408d-4335-a430-e7b8e1210981'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.712901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390c449deb4c4feb992108f500d4f8f4', 'request_id': 'req-172205d9-408d-4335-a430-e7b8e1210981'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.728373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] The token cache is being invalidate because project 390c449deb4c4feb992108f500d4f8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:14.728760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.729491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-172205d9-408d-4335-a430-e7b8e1210981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:14.731147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1139/2298] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:14 2022] DELETE /identity/v3/projects/390c449deb4c4feb992108f500d4f8f4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:14.737570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.738423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.738423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.763043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:14.764592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:14.800098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:14.835118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1160/2299] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:14.843259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.843574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.843842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.865255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:14.866466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:14.877690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1140/2300] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:14.881866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:14.891936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e4887-b19d-405c-bc85-773ae19f0153 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:14.899112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:14.899833 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.900105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.900882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.901067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:14.901387 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:14.903599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:14.917875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1161/2301] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:08:14.922235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:14.931654 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6184f41-cdee-4ee4-bf05-fb44fe3a2e53 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:14.936828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:14.937429 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:14.937704 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:14.937971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:14.938613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:14.940615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:14.969576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1141/2302] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:14 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:14.973661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:14.984545 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb6f9-5923-49f9-86ff-a6ab09f7a8a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:15.041546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:15.042157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.042427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.042692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.043417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:15.045566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:15.068457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1162/2303] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:14 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:15.072586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:15.083196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a58-d9ac-4a3c-8e9b-fc45272e1b11 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:15.087685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:15.088276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.088550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.088816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.089447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:15.091150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:15.101051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1142/2304] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:15.105074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:15.115689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b768f6-4fb2-419e-a9b4-6d5d2e0e3f16 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:15.120934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:15.121523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.121796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.122064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.122839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:15.124479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d295d14-1e46-4f85-bb2c-592b329d6b3b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:15.137413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1163/2305] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:15.141822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:15.152154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac66d1-0015-4773-9f29-d7bab5059b9a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:15.156909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:15.157548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.157816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.158102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] PATH_INFO: `/v3/projects/6157780581874b9cb74a967a899cddd8/users/3503d1081b314bf9b789d84a66e31fd5/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.173150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6157780581874b9cb74a967a899cddd8, user_id=3503d1081b314bf9b789d84a66e31fd5, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:15.175978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:15.190152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1143/2306] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:15 2022] PUT /identity/v3/projects/6157780581874b9cb74a967a899cddd8/users/3503d1081b314bf9b789d84a66e31fd5/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:08:15.203367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822248-f354-425c-a2d1-5e8f2a14fe04 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.203707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822248-f354-425c-a2d1-5e8f2a14fe04 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.204050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822248-f354-425c-a2d1-5e8f2a14fe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.237465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1c822248-f354-425c-a2d1-5e8f2a14fe04 None None] MFA Rules not processed for user `3503d1081b314bf9b789d84a66e31fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:15.239008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c822248-f354-425c-a2d1-5e8f2a14fe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:15.310925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1164/2307] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:15.322913 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:15.324673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:15.403632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:15.426846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d7b6-5dad-49e2-8961-3b6e7f6f481a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:15.431047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:15.431657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:15.431928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:15.432194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:15.440640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:15.442075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:15.451176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1144/2308] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:25.307738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:25.446846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=11 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:27.122253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e331f7-4e53-42bb-bffe-b1f5463ce009 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:27.122729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e331f7-4e53-42bb-bffe-b1f5463ce009 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:27.122729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e331f7-4e53-42bb-bffe-b1f5463ce009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:27.144453 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-69e331f7-4e53-42bb-bffe-b1f5463ce009 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:27.145679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-69e331f7-4e53-42bb-bffe-b1f5463ce009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:27.203876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1165/2309] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:27.250102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:27.251791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:27.324825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27f6f-0c49-42aa-8c31-20d9b3dde1fe service glance] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:27.330263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:27.330866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:27.331236 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:27.331437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:27.339984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:27.341618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:27.349890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1145/2310] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:08:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:08:27.936334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839d8b8-4047-4cec-a493-c7efa8db0991 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:27.936610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839d8b8-4047-4cec-a493-c7efa8db0991 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:27.936908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839d8b8-4047-4cec-a493-c7efa8db0991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:27.956206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2839d8b8-4047-4cec-a493-c7efa8db0991 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:27.957412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2839d8b8-4047-4cec-a493-c7efa8db0991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:27.965039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1166/2311] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:27.980855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:27.992578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380603-5dcf-4a17-a415-4bf27b9a21a2 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:27.998742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:27.999479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:27.999758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:28.000031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:28.009730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:28.011564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:28.020620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1146/2312] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:08:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:08:28.271515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208c42-5beb-40dc-ba57-a565530f7361 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:28.272005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208c42-5beb-40dc-ba57-a565530f7361 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:28.272125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd208c42-5beb-40dc-ba57-a565530f7361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:28.291671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cd208c42-5beb-40dc-ba57-a565530f7361 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:28.292827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cd208c42-5beb-40dc-ba57-a565530f7361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:28.302457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1167/2313] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:28.316054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:28.326721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c9a4c-ed0a-44ee-af4b-2a0eddce8e9b service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:28.334809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:28.335464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:28.335742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:28.336012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:28.344642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:28.346224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:28.354416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1147/2314] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:08:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:08:28.704957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:28.705370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:28.705794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:28.730243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:28.731950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:28.740316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1168/2315] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:28.761978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:28.773260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e7b6-9d9e-4066-b6b3-c6742ac17381 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:28.778587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:28.779527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:28.780133 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:28.780571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:28.790396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:28.792277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6820b9c-8fae-493a-9b62-40e392103554 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:28.804101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1148/2316] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:08:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:08:30.556199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:30.557771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:30.625895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682639c5-63b3-4a65-801c-fac4f94be8f3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:30.630777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:30.631442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:30.631717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:30.631990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] PATH_INFO: `/v3/users/139032a1d6d54ddd9981db349c38bbfe` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:30.633153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] RBAC: Authorizing `identity:delete_user(user_id=139032a1d6d54ddd9981db349c38bbfe)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:30.635162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:30.685385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139032a1d6d54ddd9981db349c38bbfe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:30.697824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139032a1d6d54ddd9981db349c38bbfe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:30.708049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139032a1d6d54ddd9981db349c38bbfe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:30.708960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139032a1d6d54ddd9981db349c38bbfe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:30.709672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139032a1d6d54ddd9981db349c38bbfe'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:30.724363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1169/2317] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:30 2022] DELETE /identity/v3/users/139032a1d6d54ddd9981db349c38bbfe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:30.733838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:30.734329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:30.734421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:30.755869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:30.757069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:30.828714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1149/2318] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:30.839005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:30.840603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:30.847526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:30.904108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf433-af4b-4624-ae2d-8fbb7aec507b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:30.908160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:30.908761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:30.909031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:30.909297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:30.919740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:30.921378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaa9f4-036e-4190-903e-6216aba8b717 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:30.931747 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1170/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:31.010743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:31.013889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:31.028550 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b68f0-b116-4e55-b051-ef78696b5aed None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:31.044352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:31.045811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:31.046154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:31.046492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] PATH_INFO: `/v3/users/3fa26512830547db839cea342721da4b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:31.047756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa26512830547db839cea342721da4b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:31.050075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:31.110055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:31.113405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa26512830547db839cea342721da4b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.114272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa26512830547db839cea342721da4b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.124302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa26512830547db839cea342721da4b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.140311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa26512830547db839cea342721da4b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.141165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa26512830547db839cea342721da4b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.163423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1150/2320] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:31 2022] DELETE /identity/v3/users/3fa26512830547db839cea342721da4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:31.167936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:31.168286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:31.168591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:31.189820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:31.191163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:31.261404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:31.276841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:31.323699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1171/2321] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:31.334232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:31.335671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:31.390642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112492-65d9-4b98-82fc-79f913249e76 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:31.399157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:31.399157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:31.399157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:31.399157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] PATH_INFO: `/v3/projects/6e93e81f9bb0432f8feaec2d8da85a47` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:31.402171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e93e81f9bb0432f8feaec2d8da85a47)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:31.403012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:31.441905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef5025-2170-4359-933a-ce740f86e2c5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:31.452192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:31.453104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:31.453743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:31.454173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:31.465781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:31.467500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:31.476323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e93e81f9bb0432f8feaec2d8da85a47', 'request_id': 'req-64d8aa3b-d1d5-4176-9136-00852399cc63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.477431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1172/2322] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:08:31.491618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e93e81f9bb0432f8feaec2d8da85a47', 'request_id': 'req-64d8aa3b-d1d5-4176-9136-00852399cc63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.497647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e93e81f9bb0432f8feaec2d8da85a47', 'request_id': 'req-64d8aa3b-d1d5-4176-9136-00852399cc63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.511955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] The token cache is being invalidate because project 6e93e81f9bb0432f8feaec2d8da85a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:31.512344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.512969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d8aa3b-d1d5-4176-9136-00852399cc63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.514968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1151/2323] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:31 2022] DELETE /identity/v3/projects/6e93e81f9bb0432f8feaec2d8da85a47 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:31.693034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:31.695848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:31.717194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738741e-efc8-45ef-8c2d-f28cd9dc5641 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:31.768753 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:31.778479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:31.779180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:31.779458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:31.779725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] PATH_INFO: `/v3/projects/4dceffdfa7224a4a84be50c94c6f3d4b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:31.780856 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dceffdfa7224a4a84be50c94c6f3d4b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:31.782752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:31.830152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dceffdfa7224a4a84be50c94c6f3d4b', 'request_id': 'req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.836826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dceffdfa7224a4a84be50c94c6f3d4b', 'request_id': 'req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.842552 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dceffdfa7224a4a84be50c94c6f3d4b', 'request_id': 'req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.856417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] The token cache is being invalidate because project 4dceffdfa7224a4a84be50c94c6f3d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:31.856704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.857464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:31.858966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1173/2324] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:31 2022] DELETE /identity/v3/projects/4dceffdfa7224a4a84be50c94c6f3d4b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:33.492293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:33.492782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:33.492877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:33.519971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:33.521182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:33.573590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:33.582926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1152/2325] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:08:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:33.596214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:33.597664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:33.670500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ba7b1-a0b3-4d96-951d-26143e832f2e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:33.676077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:33.676690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:33.676973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:33.677288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:33.686498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:33.688130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4aaf43-7cda-497e-8c7f-9cec64e56c8a service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:33.698012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1174/2326] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:08:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:08:43.586190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:43.694068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:56.439096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:56.440673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:56.509506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a87535-a091-4d15-a2d0-a83e98e5162b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:56.514588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:56.515235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:56.515510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:56.515777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] PATH_INFO: `/v3/users/3503d1081b314bf9b789d84a66e31fd5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:56.516954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3503d1081b314bf9b789d84a66e31fd5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:56.519047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:56.572639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3503d1081b314bf9b789d84a66e31fd5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:56.573413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3503d1081b314bf9b789d84a66e31fd5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:56.580935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3503d1081b314bf9b789d84a66e31fd5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:56.588936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3503d1081b314bf9b789d84a66e31fd5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:56.589609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3503d1081b314bf9b789d84a66e31fd5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:56.608976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1153/2327] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:08:56 2022] DELETE /identity/v3/users/3503d1081b314bf9b789d84a66e31fd5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:56.614190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:56.614697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:56.614778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:56.635066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:56.636103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:56.706612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1175/2328] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:56.717115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:56.718689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:56.787486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020291b-8d11-4e09-a439-360acf33271b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:56.791355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:56.792056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:56.792416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:56.792726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:56.804480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:56.806145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c460c1-1ba6-483d-9c13-a24e1cbcdf03 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:56.817781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1154/2329] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:08:57.151643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.153366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:57.167412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01a389-2c9f-4c51-b116-da6f81a7f02f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.173472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.174251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.174530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.174807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] PATH_INFO: `/v3/projects/6157780581874b9cb74a967a899cddd8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.176146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6157780581874b9cb74a967a899cddd8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.178238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.227677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6157780581874b9cb74a967a899cddd8', 'request_id': 'req-a353f6dd-99d2-4516-bbd3-48837f473ba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:57.234221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6157780581874b9cb74a967a899cddd8', 'request_id': 'req-a353f6dd-99d2-4516-bbd3-48837f473ba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:57.241503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6157780581874b9cb74a967a899cddd8', 'request_id': 'req-a353f6dd-99d2-4516-bbd3-48837f473ba9'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:57.255773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] The token cache is being invalidate because project 6157780581874b9cb74a967a899cddd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:08:57.256063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:57.256731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a353f6dd-99d2-4516-bbd3-48837f473ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:08:57.258268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1176/2330] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:08:57 2022] DELETE /identity/v3/projects/6157780581874b9cb74a967a899cddd8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:57.265210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.265503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.265775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.286431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:57.287716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:57.306114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:57.351330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1155/2331] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:57.359334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.359614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.359886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.381011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:57.382216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:57.390761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1177/2332] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:08:57.394979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.405191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfdc74-07fb-4659-9cf0-257fc35b6681 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.410198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.410787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.411114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.411381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.412026 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:08:57.412308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.413912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.425922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1156/2333] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:08:57.431465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.440899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bc885-9ba8-4d9f-855d-b75d8e884da1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.445869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.446457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.446729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.446995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.447671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.449700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.488329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1178/2334] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:08:57 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:08:57.494859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.506256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f88a-a85f-4104-9dac-8cf76049f337 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.566256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.566856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.567163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.567433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.568047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.569999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.593018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1157/2335] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:08:57 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:08:57.597274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.607992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3bc87-6f62-4194-a7de-6283283f9854 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.613304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.613895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.614174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.614445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.615091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.616711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.627058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1179/2336] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:57.633139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.649428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f24a2-e31f-4a9b-bc93-ac503e79ffe2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.655641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.656252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.656539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.656811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.657440 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.659086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8892068-4804-45e5-9941-5108da976cb9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.676208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1158/2337] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:08:57.677405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.687931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca657f6-e58d-4716-b02e-6744ac5ee60a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.693317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.693982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.694259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.694529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] PATH_INFO: `/v3/projects/dfefc6f232404c6ca48d073df67c3907/users/7129e53818ce43078fdea24108da5d39/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.710842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfefc6f232404c6ca48d073df67c3907, user_id=7129e53818ce43078fdea24108da5d39, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.713544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.728405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1180/2338] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:08:57 2022] PUT /identity/v3/projects/dfefc6f232404c6ca48d073df67c3907/users/7129e53818ce43078fdea24108da5d39/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:08:57.743737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c542f09-31bd-4b34-acda-2db735ed1338 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.747379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c542f09-31bd-4b34-acda-2db735ed1338 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.747694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c542f09-31bd-4b34-acda-2db735ed1338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.768644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4c542f09-31bd-4b34-acda-2db735ed1338 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:08:57.769898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4c542f09-31bd-4b34-acda-2db735ed1338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:57.833784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1159/2339] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:08:57.844058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:08:57.845910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:08:57.869492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:08:57.906560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d2b29-d654-41bc-a349-c43edd574197 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:08:57.910175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:08:57.910832 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:08:57.911138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:08:57.911596 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:08:57.920905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:08:57.922369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:08:57.932064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1181/2340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:09:02.400929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.401466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.401636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.433165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] MFA Rules not processed for user `7129e53818ce43078fdea24108da5d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:02.434710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:02.454750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:02.508108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1160/2341] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:02.513254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.523525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a518f-84e6-40a1-adf8-f61c625bd5e0 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.528680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.529301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.529581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.529853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.530497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:02.532460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:02.568636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1182/2342] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:09:02 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:09:02.569813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.581990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06768-d9ab-4ebc-9e4f-5df8e2d9eb89 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.665745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.666357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.666631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.666898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.667570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:02.669606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:02.694172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1161/2343] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:09:02 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:09:02.698527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.709822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1d72c-911b-42b5-993d-86d290f8e4a9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.714684 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.715436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.715725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.716019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.716774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:02.718422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:02.733299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1183/2344] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:09:02.742356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.771151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63a7f6-3f87-41c4-899f-858c848ae905 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.776203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.777263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.777547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.777924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] PATH_INFO: `/v3/projects/fea0a6e89574496d8f5471e7474aa37e/users/8364f1360e944acb86d403c7cd1d9d54/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.800585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea0a6e89574496d8f5471e7474aa37e, user_id=8364f1360e944acb86d403c7cd1d9d54, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:02.803703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:02.835069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1162/2345] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:09:02 2022] PUT /identity/v3/projects/fea0a6e89574496d8f5471e7474aa37e/users/8364f1360e944acb86d403c7cd1d9d54/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:09:02.836736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.850069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1abcfb-90f9-42f6-b0bd-6c28198fadfe admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.917657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.918549 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:02.919003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:02.919393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:02.920132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:02.921820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9bb02-b1a6-4536-a07d-347c2abe6fc6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:02.933263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1184/2346] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:09:02.939975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:02.980239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f4a2e-761b-4ad5-9dc9-cd0890662a26 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:02.998116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:02.999018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:03.000561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:03.001600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] PATH_INFO: `/v3/projects/fea0a6e89574496d8f5471e7474aa37e/users/8364f1360e944acb86d403c7cd1d9d54/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:03.007423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea0a6e89574496d8f5471e7474aa37e, user_id=8364f1360e944acb86d403c7cd1d9d54, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:03.015097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67ac4e-1cd0-49f3-9733-99b8789567d6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:03.042810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1163/2347] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:09:02 2022] PUT /identity/v3/projects/fea0a6e89574496d8f5471e7474aa37e/users/8364f1360e944acb86d403c7cd1d9d54/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:09:04.237009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:04.237905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:04.238067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:04.241418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1185/2348] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:09:04.259834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b04eba-f9d3-4eed-af3b-923132209614 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:04.260397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b04eba-f9d3-4eed-af3b-923132209614 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:04.260516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b04eba-f9d3-4eed-af3b-923132209614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:04.313082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a6b04eba-f9d3-4eed-af3b-923132209614 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:04.322662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a6b04eba-f9d3-4eed-af3b-923132209614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:04.521851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1164/2349] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:09:04.522511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:04.524664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:04.667323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216482f1-cf93-4600-a2ea-c970e1e97365 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:04.675340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:04.676033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:04.676318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:04.676591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:04.694594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:04.698045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127653c-2082-4c52-959e-13a7071f2454 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:04.715857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1186/2350] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:09:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:09:04.872186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:04.873922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:04.875055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:04.879442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1165/2351] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:09:04 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:09:04.886230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ac6c-b22f-4531-bbd3-c435693884ae None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:04.886853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ac6c-b22f-4531-bbd3-c435693884ae None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:04.887439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ac6c-b22f-4531-bbd3-c435693884ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:04.915315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4e73ac6c-b22f-4531-bbd3-c435693884ae None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:04.917165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4e73ac6c-b22f-4531-bbd3-c435693884ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:04.966061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:05.060827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1187/2352] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:09:05.084886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:05.087218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:05.104519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:05.125572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c4d97-29df-439b-9297-7c80b4cc05b3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:05.140116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:05.140754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:05.141029 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:05.141295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:05.180874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:05.184596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0fdd8-ffec-4672-8492-f8ac87f0164c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:05.245814 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1166/2353] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:09:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:09:05.529015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedaff3-06fa-4af6-a7e6-99edae861f03 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:05.529580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedaff3-06fa-4af6-a7e6-99edae861f03 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:05.529885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedaff3-06fa-4af6-a7e6-99edae861f03 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:05.532268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1188/2354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 04:09:05.539759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417351b-7b3a-471b-ab52-36d4014fd2f5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:05.540621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417351b-7b3a-471b-ab52-36d4014fd2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:05.541124 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417351b-7b3a-471b-ab52-36d4014fd2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:05.597320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a417351b-7b3a-471b-ab52-36d4014fd2f5 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:05.599147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a417351b-7b3a-471b-ab52-36d4014fd2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:05.643448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1167/2355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:09:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:05.667996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedaff3-06fa-4af6-a7e6-99edae861f03 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:05.696056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedaff3-06fa-4af6-a7e6-99edae861f03 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:05.704923 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:05.705628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:05.705998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:05.706395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:05.731688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:05.733456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a0a18-cddc-4c84-8301-59646f5fdeaf service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:05.750484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1189/2356] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:09:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:09:07.898365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:07.898822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:07.898901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:07.947900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] MFA Rules not processed for user `f80c8c1d6af7450eb8f936df231ad63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:07.949504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:08.068185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1168/2357] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:10.032651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8bc-0f36-414e-b73d-7329161201e3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:10.035302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8bc-0f36-414e-b73d-7329161201e3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:10.036431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bc8bc-0f36-414e-b73d-7329161201e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:10.099669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8d2bc8bc-0f36-414e-b73d-7329161201e3 None None] MFA Rules not processed for user `8364f1360e944acb86d403c7cd1d9d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:10.101130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8d2bc8bc-0f36-414e-b73d-7329161201e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:10.221623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1190/2358] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:10.236683 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:10.256684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f2730-ca49-4c76-93df-bd523cc7c05f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:10.274463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:10.275236 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:10.276457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:10.277183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:10.383462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:10.384732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:10.396389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1169/2359] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:09:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:09:10.577553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73678e02-ae7a-42b3-a905-0d9b2a569404 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:10.577841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73678e02-ae7a-42b3-a905-0d9b2a569404 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:10.578113 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73678e02-ae7a-42b3-a905-0d9b2a569404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:10.617987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-73678e02-ae7a-42b3-a905-0d9b2a569404 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:10.621055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-73678e02-ae7a-42b3-a905-0d9b2a569404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:10.703014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1191/2360] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:09:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:10.737763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:10.739334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:10.866412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94da38-b30e-466f-83bd-7e3ab8158a02 service glance] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:10.876077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:10.877011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:10.877330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:10.877603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:10.888223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:10.889268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:10.912109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1170/2361] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:09:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:09:11.752785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5ac53-cc50-4946-be68-af40b198b0b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:11.753426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5ac53-cc50-4946-be68-af40b198b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:11.753907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5ac53-cc50-4946-be68-af40b198b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:11.801329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5da5ac53-cc50-4946-be68-af40b198b0b5 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:11.802716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5da5ac53-cc50-4946-be68-af40b198b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:11.820776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1192/2362] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:09:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:11.860672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:11.883497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f984-6db9-40e6-b19a-b9329b439fb4 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:11.896383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:11.897214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:11.897638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:11.898708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:11.917995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:11.919871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8c8f8-a279-4775-9174-8b004574fc83 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:11.944026 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1171/2363] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:09:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:09:14.283157 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c807b-4b33-48fc-a2d1-f60463e3260f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:14.283771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c807b-4b33-48fc-a2d1-f60463e3260f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:14.299211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c807b-4b33-48fc-a2d1-f60463e3260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:14.347227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8f0c807b-4b33-48fc-a2d1-f60463e3260f None None] MFA Rules not processed for user `77fb76da34bf44a1b214a08bb4b8e51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:14.348745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8f0c807b-4b33-48fc-a2d1-f60463e3260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:14.448143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:14.466318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1193/2364] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:20.503524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a14cb6-9168-41da-98d8-90b18edea3e8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:20.504455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a14cb6-9168-41da-98d8-90b18edea3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:20.504919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a14cb6-9168-41da-98d8-90b18edea3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:20.513471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:20.545789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-99a14cb6-9168-41da-98d8-90b18edea3e8 None None] MFA Rules not processed for user `ffdeb3e2f0ec411682815e1d4197f662`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:20.547172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-99a14cb6-9168-41da-98d8-90b18edea3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:20.694399 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1172/2365] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:24.039976 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a88af-219b-4662-97ce-294354874d1a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:24.041333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a88af-219b-4662-97ce-294354874d1a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:24.041971 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a88af-219b-4662-97ce-294354874d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:24.068760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-983a88af-219b-4662-97ce-294354874d1a None None] MFA Rules not processed for user `7129e53818ce43078fdea24108da5d39`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:24.070854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-983a88af-219b-4662-97ce-294354874d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:24.085525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1194/2366] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:09:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:28.678364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:28.678931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:28.679805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:28.700219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:28.701589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:28.710011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1173/2367] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:28.733070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a88af-219b-4662-97ce-294354874d1a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:28.746957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a88af-219b-4662-97ce-294354874d1a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:28.752306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:28.752899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:28.753175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:28.753444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:28.762795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:28.764428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16970fc3-2301-4746-a58f-fb73a309d7c6 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:28.773247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1195/2368] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:09:37.923157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:37.925266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:38.019586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de107a0-e5f2-4e58-a233-4b5658473b79 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:38.025391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:38.026446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:38.026949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:38.027624 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:38.029656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:38.043177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:38.045285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:38.057075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1174/2369] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:09:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:09:38.769406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=12 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:46.100919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:46.101528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:46.101528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:46.129458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:09:46.130700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:09:46.139888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1196/2370] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:09:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:09:46.153461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:09:46.164765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c092-e0fd-465e-a6ce-e6ecbfe73e73 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:09:46.170689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:09:46.171326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:09:46.171581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:09:46.172105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:09:46.182175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:09:46.183972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:09:46.192342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1175/2371] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:09:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:09:56.124281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:09:56.189166 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=65 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:07.185969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:07.200760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91512b4-f1db-477b-ba42-2669f88d767f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:07.205850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:07.206469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:07.206749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:07.207021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:07.219219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:07.220691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:07.232370 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1197/2372] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:10:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:10:17.226896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:39.620825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:39.622666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:39.693341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20417-1647-469d-be5e-c65d9dccbb89 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:39.698183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:39.698798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:39.699077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:39.699380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] PATH_INFO: `/v3/users/ffdeb3e2f0ec411682815e1d4197f662` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:39.700607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdeb3e2f0ec411682815e1d4197f662)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:39.702566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:39.761292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdeb3e2f0ec411682815e1d4197f662'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:39.762277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdeb3e2f0ec411682815e1d4197f662'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:39.771818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdeb3e2f0ec411682815e1d4197f662'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:39.785076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdeb3e2f0ec411682815e1d4197f662'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:39.785964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6758a5e-87ed-448c-94b4-ff92e07544e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdeb3e2f0ec411682815e1d4197f662'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:39.807522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1176/2373] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:10:39 2022] DELETE /identity/v3/users/ffdeb3e2f0ec411682815e1d4197f662 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:10:39.998911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.017097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57642664-ad05-4f45-9f25-1f1d0fb13130 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.083173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.083847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.084164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.084336 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] PATH_INFO: `/v3/projects/d362e24b36234e6b9a62f07a06dcd9f3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.085469 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d362e24b36234e6b9a62f07a06dcd9f3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.087428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.143065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd362e24b36234e6b9a62f07a06dcd9f3', 'request_id': 'req-94c40617-fca4-429d-86e2-b027988f883c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:40.150455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd362e24b36234e6b9a62f07a06dcd9f3', 'request_id': 'req-94c40617-fca4-429d-86e2-b027988f883c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:40.159669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd362e24b36234e6b9a62f07a06dcd9f3', 'request_id': 'req-94c40617-fca4-429d-86e2-b027988f883c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:40.174001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] The token cache is being invalidate because project d362e24b36234e6b9a62f07a06dcd9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:10:40.174280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:40.174877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-94c40617-fca4-429d-86e2-b027988f883c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:10:40.176395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1198/2374] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:10:39 2022] DELETE /identity/v3/projects/d362e24b36234e6b9a62f07a06dcd9f3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:10:40.181582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.181873 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.182151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.202614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:40.203860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:40.265431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1177/2375] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:10:40.333333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.333611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.333859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.363999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:40.365879 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:40.378098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1199/2376] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:10:40.382582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.393725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb4bfd-a6d0-4a38-b243-6d4250fda62c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.399705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.400296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.400571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.400831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.401478 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:10:40.401773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.403454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.416444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1178/2377] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:10:40.421550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.431470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334230b-de86-483d-b6bd-22875ef7ca6e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.437754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.438352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.438626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.438890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.439663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.441587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.478797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1200/2378] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:10:40 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:10:40.483982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.494985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9eb25-e300-4b7d-9010-1422fef68267 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.499061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:40.556323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.556923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.557197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.557463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.558092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.560102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.587237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1179/2379] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:10:40 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:10:40.588600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.599270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b075c-81d7-4f6c-bee0-43539f2274d4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.604220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.604882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.605158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.605425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.606070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.607852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.620635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1201/2380] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:10:40.625125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.637024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0256-31ca-49c1-b662-2cd58db27c19 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.642419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.643079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.643367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.643640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.644279 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.645900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0db49-a9f3-4b96-8ecb-420a55bc4240 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.657329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1180/2381] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:10:40.661585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.672573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2011b-053e-460e-9c25-ade8f3db7217 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.678071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.678754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.679016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.679337 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] PATH_INFO: `/v3/projects/e6db5b66c75c499d98d02d6e0041299f/users/62f5d4659eea4d91a689d709460396d2/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.694663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6db5b66c75c499d98d02d6e0041299f, user_id=62f5d4659eea4d91a689d709460396d2, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.697405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.715772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1202/2382] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:10:40 2022] PUT /identity/v3/projects/e6db5b66c75c499d98d02d6e0041299f/users/62f5d4659eea4d91a689d709460396d2/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:10:40.721073 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a352a-c36f-41ad-be41-8a385c81e0bb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.721362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a352a-c36f-41ad-be41-8a385c81e0bb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.721649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a352a-c36f-41ad-be41-8a385c81e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.744392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f76a352a-c36f-41ad-be41-8a385c81e0bb None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:40.745611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f76a352a-c36f-41ad-be41-8a385c81e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:40.808195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1181/2383] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:10:40.822937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:40.824846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:40.857005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:40.886298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a1e49-0b85-4d7a-8d53-5d04dd5bf2a5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:40.890301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:40.890924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:40.891231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:40.891945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:40.902091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:40.903654 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:40.915380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1203/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:10:48.256462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.256995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.257078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.304674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] MFA Rules not processed for user `62f5d4659eea4d91a689d709460396d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:48.306159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:48.411116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1182/2385] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:10:48.411228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.423430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f1a0c-c451-4d80-8582-eccc60da3d1e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.428425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.429035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.429313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.429731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.430483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.432456 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.471082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1204/2386] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:10:48 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:10:48.472366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.485598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3be4f5-8337-4ddb-8aef-4f6f421c02fa None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.499821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:48.559394 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.560099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.560358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.560887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.561568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.563606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.591693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1183/2387] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:10:48 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:10:48.595917 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.608477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d90a6-a1fc-4410-bea1-5046db53cd7a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.614272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.614975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.615362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.615724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.616445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.618120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.634527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1205/2388] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:10:48.635792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.649984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2575f-e275-493d-ae8b-d2acc5ad52a1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.664035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.664035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.664035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.664035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] PATH_INFO: `/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a/users/a66f90f59af641da976dde35d8b4394b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.681714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46652dc4c6e940c3a1dbb2c7dc057f3a, user_id=a66f90f59af641da976dde35d8b4394b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.685610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.718276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1184/2389] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:10:48 2022] PUT /identity/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a/users/a66f90f59af641da976dde35d8b4394b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:10:48.721642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.736542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27427a-fdf3-4409-8dfe-7267aca031bb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.807486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.808120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.808399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.808673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.809468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.811289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ffe2a-23f0-4c4c-a3e3-68e70e691250 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.823444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1206/2390] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:10:48.827852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:48.842727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de42242-59a0-457a-a5ae-e61436d6d496 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:48.848644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:48.849414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:48.849646 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:48.849923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] PATH_INFO: `/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a/users/a66f90f59af641da976dde35d8b4394b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:48.852107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46652dc4c6e940c3a1dbb2c7dc057f3a, user_id=a66f90f59af641da976dde35d8b4394b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:48.854833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537e961-9b7e-43f7-ad0a-b116cebcc5eb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:48.868997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1185/2391] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:10:48 2022] PUT /identity/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a/users/a66f90f59af641da976dde35d8b4394b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:10:50.878939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:50.879537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:50.879537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:50.911258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] MFA Rules not processed for user `956b08da3ba944d382877cf266389127`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:50.917998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:51.047516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1207/2392] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:10:58.162901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a9e5-a35e-4196-ad64-83d0548efd54 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:58.163518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a9e5-a35e-4196-ad64-83d0548efd54 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:58.163518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a9e5-a35e-4196-ad64-83d0548efd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:58.226300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9e22a9e5-a35e-4196-ad64-83d0548efd54 None None] MFA Rules not processed for user `a66f90f59af641da976dde35d8b4394b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:10:58.228591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9e22a9e5-a35e-4196-ad64-83d0548efd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:58.394701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1186/2393] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:10:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:10:58.412696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:10:58.414979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:10:58.528892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:10:58.597106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107cc2c-60ae-4aa4-9eb9-fc0f4fa220af None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:10:58.604726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:10:58.605518 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:10:58.606003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:10:58.606356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:10:58.716697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:10:58.719503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2d69-2c0f-47e8-ad0b-214ee0ed780f service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:10:58.751554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1208/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:10:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:11:02.606694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:02.607393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:02.607393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:02.628733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] MFA Rules not processed for user `f80c8c1d6af7450eb8f936df231ad63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:11:02.629972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:02.680848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:02.692381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1187/2395] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:11:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:11:08.730077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=39 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:12.691548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:13.530999 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bb7a-206e-479d-81e7-1ef37fca8807 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:13.531580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bb7a-206e-479d-81e7-1ef37fca8807 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:13.531695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bb7a-206e-479d-81e7-1ef37fca8807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:13.557095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a6f5bb7a-206e-479d-81e7-1ef37fca8807 None None] MFA Rules not processed for user `77fb76da34bf44a1b214a08bb4b8e51b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:11:13.558356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a6f5bb7a-206e-479d-81e7-1ef37fca8807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:13.630590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1209/2396] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:11:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:11:16.038991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:16.102916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9590f1-530c-4664-a3fb-bb104259eeb1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:16.107182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:16.107787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:16.108062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:16.108315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:16.110791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:16.124602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:16.126256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:16.137293 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1188/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:11:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:11:16.401355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:16.401817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:16.401933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:16.423364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] MFA Rules not processed for user `62f5d4659eea4d91a689d709460396d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:11:16.424827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:16.432262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1210/2398] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:11:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:11:17.294815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:17.304923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03308-0f8c-4940-9dc8-d33d67b21721 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:17.308883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:17.309498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:17.310706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:17.310969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:17.312916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:17.328465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:17.330417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697043aa-a8db-4677-a6b9-d89b9f767b87 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:17.345587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1189/2399] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:11:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:11:26.417725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=28 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:27.337500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=32 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:34.574255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:34.585015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389282f2-ac77-4243-87dd-0b2e66b6bbfd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:34.589259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:34.589860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:34.590139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:34.590409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:34.600302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:34.601744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:34.612085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1211/2400] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:11:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:11:44.607126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:45.953879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:45.954341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:45.954442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:45.974390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:11:45.975629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:46.036889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1190/2401] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:11:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:11:46.088326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:46.089857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:46.164559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f70609-9976-4bb6-8985-e7da5dc6e282 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:46.170031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:46.170620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:46.170898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:46.171142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:46.179537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:46.181152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:46.189126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1212/2402] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:11:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:11:52.233195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:52.235412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:52.252106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068139e-dbad-41e5-857b-745f87b9f6aa None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:52.260370 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:52.260980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:52.261446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:52.261887 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:52.264249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:52.283170 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:52.285177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:52.296831 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1191/2403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:11:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:11:53.339075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:53.341368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:53.429406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b885e2-c4d1-446b-b5ac-fa748ce24493 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:53.434605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:53.435287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:53.435709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:53.435836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] PATH_INFO: `/v3/users/7129e53818ce43078fdea24108da5d39` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:53.437294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7129e53818ce43078fdea24108da5d39)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:53.439260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:53.488535 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7129e53818ce43078fdea24108da5d39'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.501432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7129e53818ce43078fdea24108da5d39'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.509247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7129e53818ce43078fdea24108da5d39'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.510004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7129e53818ce43078fdea24108da5d39'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.510686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7129e53818ce43078fdea24108da5d39'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.527286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1213/2404] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:11:53 2022] DELETE /identity/v3/users/7129e53818ce43078fdea24108da5d39 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:11:53.709868 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:53.722295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b02fc4-3371-407a-af12-6f0623ad5a45 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:53.785450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:53.786097 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:53.786357 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:53.786619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] PATH_INFO: `/v3/projects/dfefc6f232404c6ca48d073df67c3907` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:53.787986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfefc6f232404c6ca48d073df67c3907)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:53.790383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:53.844019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfefc6f232404c6ca48d073df67c3907', 'request_id': 'req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.850572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfefc6f232404c6ca48d073df67c3907', 'request_id': 'req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.856404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfefc6f232404c6ca48d073df67c3907', 'request_id': 'req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.871572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] The token cache is being invalidate because project dfefc6f232404c6ca48d073df67c3907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:11:53.871858 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.872475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:53.873973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1192/2405] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:11:53 2022] DELETE /identity/v3/projects/dfefc6f232404c6ca48d073df67c3907 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:11:53.878219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:53.879860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:53.893071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed2825-492c-4298-bcc9-c6bde8a0f253 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:53.898135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:53.951994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:53.952513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:53.952786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:53.953059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] PATH_INFO: `/v3/users/8364f1360e944acb86d403c7cd1d9d54` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:53.954198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8364f1360e944acb86d403c7cd1d9d54)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:53.956165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:54.010269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8364f1360e944acb86d403c7cd1d9d54'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.022263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8364f1360e944acb86d403c7cd1d9d54'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.029155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8364f1360e944acb86d403c7cd1d9d54'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.029814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8364f1360e944acb86d403c7cd1d9d54'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.030412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8364f1360e944acb86d403c7cd1d9d54'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.046001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1214/2406] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:11:53 2022] DELETE /identity/v3/users/8364f1360e944acb86d403c7cd1d9d54 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:11:54.224699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:11:54.226150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:11:54.244008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c55bd-2b35-4e84-b2ba-586ecb2abbcf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:11:54.263910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:11:54.307811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:11:54.308461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:11:54.308741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:11:54.309013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] PATH_INFO: `/v3/projects/fea0a6e89574496d8f5471e7474aa37e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:11:54.310154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea0a6e89574496d8f5471e7474aa37e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:11:54.312119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:11:54.373768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea0a6e89574496d8f5471e7474aa37e', 'request_id': 'req-c588ea59-3a8e-48b2-96f1-72ae07a3a381'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.382809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea0a6e89574496d8f5471e7474aa37e', 'request_id': 'req-c588ea59-3a8e-48b2-96f1-72ae07a3a381'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.389691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea0a6e89574496d8f5471e7474aa37e', 'request_id': 'req-c588ea59-3a8e-48b2-96f1-72ae07a3a381'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.405619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] The token cache is being invalidate because project fea0a6e89574496d8f5471e7474aa37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:11:54.405909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.406592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:11:54.408218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1193/2407] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:11:54 2022] DELETE /identity/v3/projects/fea0a6e89574496d8f5471e7474aa37e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:00.290017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:00.291651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:00.361494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0853-4e08-4156-b549-465308cd762d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:00.409399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:00.419616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:00.420252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:00.420686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:00.421186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:00.423230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:00.493361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:00.495511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:00.505251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1215/2408] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:02.785780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:02.788749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:02.874836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ea59-3a8e-48b2-96f1-72ae07a3a381 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:02.881298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:02.881881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:02.882149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:02.882441 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:02.884473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:02.934384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:02.945562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:02.947585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:02.957520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1194/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:06.664569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:06.677870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c4fe-4c26-4eb1-add8-20e301de1103 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:06.683551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:06.684183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:06.684448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:06.684709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:06.686486 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:06.699229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:06.700841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d02464-17b6-412b-8fe8-fc27160f2468 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:06.722553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1216/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:12.956318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:14.015572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:14.029864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bd19a-18e5-4b4f-aec6-f52f306ad822 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:14.035481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:14.036084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:14.036361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:14.036628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:14.038590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:14.050981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:14.052608 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:14.064195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1195/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:14.869973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:14.870959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:14.871569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:14.892442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:14.893656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:14.960338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1217/2412] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:12:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:14.974524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:14.976531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:15.049045 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689ab15-73c2-4272-abe9-960e119295ad service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:15.054605 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:15.055329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:15.055710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:15.055871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:15.065059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:15.066463 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:15.075770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1196/2413] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:12:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:12:21.643050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:21.645547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:21.712122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e34e4-1a7a-4fba-b579-24a773b2db3e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:21.715964 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:21.716581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:21.716990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:21.718322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:21.721180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:21.789222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:21.813645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:21.816434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:21.827931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1218/2414] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:25.072137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=38 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:25.290173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:25.301992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b835830-ca4a-4a84-bb41-0c54f006dbe9 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:25.307505 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:25.308318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:25.309364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:25.309795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:25.313710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:25.337749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:25.341275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:25.388580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1197/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:26.717640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:26.719899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:26.733220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d134b-7e63-4777-879a-07a2808de714 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:26.739080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:26.739898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:26.740233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:26.740645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] PATH_INFO: `/v3/users/77fb76da34bf44a1b214a08bb4b8e51b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:26.741950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77fb76da34bf44a1b214a08bb4b8e51b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:26.744260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:26.799748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77fb76da34bf44a1b214a08bb4b8e51b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:26.814593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77fb76da34bf44a1b214a08bb4b8e51b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:26.825139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77fb76da34bf44a1b214a08bb4b8e51b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:26.826043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77fb76da34bf44a1b214a08bb4b8e51b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:26.826766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b63843fa-09af-4b9b-aa52-cae6be5d8470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77fb76da34bf44a1b214a08bb4b8e51b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:26.846703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1219/2416] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:26 2022] DELETE /identity/v3/users/77fb76da34bf44a1b214a08bb4b8e51b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:27.055668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.068876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd67b8c-bdf9-4575-a425-d657efe2fbec service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.136462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.137273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.137734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.138146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] PATH_INFO: `/v3/projects/490f695240294bbaa99f5eaa1e32b977` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.139515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490f695240294bbaa99f5eaa1e32b977)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.141758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.191261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490f695240294bbaa99f5eaa1e32b977', 'request_id': 'req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:27.197372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490f695240294bbaa99f5eaa1e32b977', 'request_id': 'req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:27.202845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490f695240294bbaa99f5eaa1e32b977', 'request_id': 'req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:27.215800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] The token cache is being invalidate because project 490f695240294bbaa99f5eaa1e32b977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:27.216085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:27.216716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ef622f0-d80b-4d1a-9b43-7a6d435a9269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:27.220329 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1198/2417] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:27 2022] DELETE /identity/v3/projects/490f695240294bbaa99f5eaa1e32b977 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:27.223270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.223626 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.223961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.244676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:27.245906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:27.323501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1220/2418] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:27.331716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.332057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.338604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.361588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:27.364477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:27.376115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1199/2419] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:27.380853 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.392657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ff493-ba82-4d60-937e-f3a3018739e4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.398286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.399301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.399846 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.400195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.400921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:12:27.401281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.402965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.415986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1221/2420] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:12:27.420122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.429694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167eee1-baa3-4c17-9bd3-c2770c62498e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.434447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.435259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.435817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.436091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.438016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.440003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.473666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1200/2421] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:27.474862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.485538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66699a-f11e-4f96-a438-41071bff8848 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.529714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:27.555554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.556171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.556447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.556715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.557349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.559686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.587220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1222/2422] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:27.591996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.607692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dda3bd-6ebf-41c8-9ae1-0a7ae3fa8321 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.613562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.614516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.614950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.615371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.616562 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.617799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.631445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1201/2423] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:27.633988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.646642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade099-6b0c-426b-aa8c-ea8692d77b3d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.652807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.653312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.653595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.653869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.654815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.656750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f77c-40a3-4ace-9b02-d69761e59942 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.671350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1223/2424] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:27.675937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.689966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63895-a493-4a7c-88ff-b9c058c23a67 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.695695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.696639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.696987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.703059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] PATH_INFO: `/v3/projects/25ff2590d9b34b97acfa2bde97840410/users/b9752cc3ec9a4262844a8783f92af2ac/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.719876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ff2590d9b34b97acfa2bde97840410, user_id=b9752cc3ec9a4262844a8783f92af2ac, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.722555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.738697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1202/2425] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:27 2022] PUT /identity/v3/projects/25ff2590d9b34b97acfa2bde97840410/users/b9752cc3ec9a4262844a8783f92af2ac/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:27.751214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.751751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.751751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.783604 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] MFA Rules not processed for user `b9752cc3ec9a4262844a8783f92af2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:27.784962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:27.856360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1224/2426] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:27.861649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.877609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:27.936039 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8f6f4-0aac-4833-a4a6-5887f4cf4d64 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:27.941519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:27.942437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:27.942864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:27.943633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:27.944287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:27.946378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661bc65-afdd-46c8-b3a6-7356e3c6b1d7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:27.980800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1203/2427] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:27.985317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:27.997778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f20bd2-d514-4917-a1c4-2b1bd6b3e745 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.019148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.019398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.019665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.059239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] MFA Rules not processed for user `f80c8c1d6af7450eb8f936df231ad63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:28.060506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.086092 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.086983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.087464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.088023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.088919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.092273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.131445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1225/2428] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:27 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:28.132554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.143447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9a856-e3a9-40d4-83ef-9aeb0456dfd3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.149273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.150056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.150221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1204/2429] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:28.154772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.154772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.154772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.154772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.167563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1226/2430] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:28.172012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.183522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a12972-8ad0-4f8a-90b2-9f514c7b90a2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.188386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.188986 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.189266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.189534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] PATH_INFO: `/v3/projects/0301db68f2be49648cd51f9bfc01eba3/users/d0d91820bd4c4da1b631db259dbf07fc/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.214066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0301db68f2be49648cd51f9bfc01eba3, user_id=d0d91820bd4c4da1b631db259dbf07fc, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.217678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.232542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1227/2431] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:28 2022] PUT /identity/v3/projects/0301db68f2be49648cd51f9bfc01eba3/users/d0d91820bd4c4da1b631db259dbf07fc/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:28.237036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.248655 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ec8c5-fc7a-4d33-b035-0f21428d83ad None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.312732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.313588 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.313867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.314163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.314824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.316609 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dff23-e250-4cf4-923d-e7bd832df99d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.327269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1205/2432] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:28.331501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.337911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:28.343819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7cc6-7e85-424a-9c5a-dd920a013caf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.349760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.350573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.350848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.351158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] PATH_INFO: `/v3/projects/0301db68f2be49648cd51f9bfc01eba3/users/d0d91820bd4c4da1b631db259dbf07fc/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.353077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0301db68f2be49648cd51f9bfc01eba3, user_id=d0d91820bd4c4da1b631db259dbf07fc, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.355904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.367700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1228/2433] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:28 2022] PUT /identity/v3/projects/0301db68f2be49648cd51f9bfc01eba3/users/d0d91820bd4c4da1b631db259dbf07fc/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:28.378155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.378433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.378729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.408774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] MFA Rules not processed for user `d0d91820bd4c4da1b631db259dbf07fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:28.410096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.442787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:28.488398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1206/2434] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:28.493468 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.495492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.502371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.505092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.567194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a4878-f2e6-4407-bc9c-e36a90fe8811 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.571488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.572111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.572388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.572657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.573629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4a6df-1cdd-4192-a0db-3bffa6fcc2c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.574537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.577290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.577898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.578174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.578447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.585298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.586764 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.587940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.589396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfe8a0-b88c-496a-8718-d21ddfa902bb service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.595014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1229/2435] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:28.599043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1207/2436] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:28.630442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.697597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de8351-728d-4567-a440-7c63e259819e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:28.702639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:28.703284 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.703564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.703834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] PATH_INFO: `/v3/users/b9752cc3ec9a4262844a8783f92af2ac` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.704985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9752cc3ec9a4262844a8783f92af2ac)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:28.706926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:28.764074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9752cc3ec9a4262844a8783f92af2ac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:28.777862 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9752cc3ec9a4262844a8783f92af2ac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:28.787507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9752cc3ec9a4262844a8783f92af2ac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:28.788347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9752cc3ec9a4262844a8783f92af2ac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:28.788987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9752cc3ec9a4262844a8783f92af2ac'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:28.805575 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1230/2437] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:28 2022] DELETE /identity/v3/users/b9752cc3ec9a4262844a8783f92af2ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:28.810281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:28.810564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:28.810856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:28.830497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:28.831733 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.907767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1208/2438] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:28.918415 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:28.920170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:28.938373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:29.001779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:29.003799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:29.005078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222882f6-b76f-455b-a83b-4ab50570ab7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:29.012602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:29.013251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:29.013641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:29.014052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:29.025902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:29.027527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:29.038080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1231/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:29.108138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009143bb-40f7-47ec-aa79-9b1394e2b91e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:29.114825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:29.115557 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:29.116009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:29.116473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:29.118830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:29.133585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:29.136049 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:29.141052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:29.150934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1209/2440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:12:29.352584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:29.354019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:29.372930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36344-e801-4e18-bed2-7daef133c75e service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:29.379824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:29.380952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:29.381267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:29.381796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] PATH_INFO: `/v3/projects/25ff2590d9b34b97acfa2bde97840410` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:29.384081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ff2590d9b34b97acfa2bde97840410)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:29.386522 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:29.454873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ff2590d9b34b97acfa2bde97840410', 'request_id': 'req-acf10b04-d722-41d7-bacd-438d45bc939e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.462969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ff2590d9b34b97acfa2bde97840410', 'request_id': 'req-acf10b04-d722-41d7-bacd-438d45bc939e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.469751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ff2590d9b34b97acfa2bde97840410', 'request_id': 'req-acf10b04-d722-41d7-bacd-438d45bc939e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.486016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] The token cache is being invalidate because project 25ff2590d9b34b97acfa2bde97840410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:29.486560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.487187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.491233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1232/2441] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:29 2022] DELETE /identity/v3/projects/25ff2590d9b34b97acfa2bde97840410 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:29.492818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:29.504958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a8e3c-5ed4-4fcd-a148-06335f1d76fb service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:29.571225 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:29.571857 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:29.572116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:29.572389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] PATH_INFO: `/v3/users/d0d91820bd4c4da1b631db259dbf07fc` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:29.574382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d91820bd4c4da1b631db259dbf07fc)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:29.576385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:29.640121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d91820bd4c4da1b631db259dbf07fc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.640938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d91820bd4c4da1b631db259dbf07fc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.651568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d91820bd4c4da1b631db259dbf07fc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.660846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d91820bd4c4da1b631db259dbf07fc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.661750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d91820bd4c4da1b631db259dbf07fc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:29.681619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1210/2442] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:29 2022] DELETE /identity/v3/users/d0d91820bd4c4da1b631db259dbf07fc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:29.896033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:29.897555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:29.959079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10b04-d722-41d7-bacd-438d45bc939e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:29.991981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:29.993771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.012699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.013314 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.013594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.014054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.016027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.023946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:30.026614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.028773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b25e6-c9ca-4431-ac8d-aee136194bed service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.039156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1233/2443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:30.069434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6c29-e30e-4c63-9ec7-07143421a9ff admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.074590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.075253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.075621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.075903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] PATH_INFO: `/v3/projects/0301db68f2be49648cd51f9bfc01eba3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.077138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0301db68f2be49648cd51f9bfc01eba3)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.079103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.102152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:30.153010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0301db68f2be49648cd51f9bfc01eba3', 'request_id': 'req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:30.163644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0301db68f2be49648cd51f9bfc01eba3', 'request_id': 'req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:30.178011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0301db68f2be49648cd51f9bfc01eba3', 'request_id': 'req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:30.196034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1aee-bfa8-49d5-bd90-33c788df172a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.196607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1aee-bfa8-49d5-bd90-33c788df172a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.197037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b1aee-bfa8-49d5-bd90-33c788df172a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.198614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1234/2444] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:12:30.202986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] The token cache is being invalidate because project 0301db68f2be49648cd51f9bfc01eba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:30.204242 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:30.205292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3a9612f4-ee05-4f47-9d4c-f99dd237b1c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:30.206804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1211/2445] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:29 2022] DELETE /identity/v3/projects/0301db68f2be49648cd51f9bfc01eba3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:30.212272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18203-e120-4af0-af63-906c1966eaf2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.212629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18203-e120-4af0-af63-906c1966eaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.213098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18203-e120-4af0-af63-906c1966eaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.214020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.214412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.214691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.235193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d2e18203-e120-4af0-af63-906c1966eaf2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.236570 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d2e18203-e120-4af0-af63-906c1966eaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.238299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.239707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.306030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1212/2446] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:30.313602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.313886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.314157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.337031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1235/2447] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:30.354985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.356786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.359705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.361754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.382644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1213/2448] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:30.384015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.393539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60202ba1-f8ea-4eee-8404-4db7241586c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.404861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.405737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.406108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.406663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.408405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:12:30.409495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.412910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.427577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515e5fb-1a57-451f-a536-dec9808681fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.432784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1214/2449] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:12:30.433808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.446161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04a89c-97d6-4ba0-a399-747fb80e3cb9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.446405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.448509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.448797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.449162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.453074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.454175 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.454923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.455567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.456456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.461785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.474218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.477473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4ab88-7058-4042-a879-73f959f4d704 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.501965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1215/2450] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:30.506384 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1236/2451] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:30.508602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.602888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d1cfe-3cac-47d5-8c5d-e8cd8e17807a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.605848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882cedc-a398-413e-b59e-21c9551915ed None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.606133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882cedc-a398-413e-b59e-21c9551915ed None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.606422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882cedc-a398-413e-b59e-21c9551915ed None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.608551 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1237/2452] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:12:30.609030 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.609714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.610065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.610346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.610983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.613363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.613591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20980515-baad-4664-8b30-8906895959ca None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.613960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20980515-baad-4664-8b30-8906895959ca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.614514 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20980515-baad-4664-8b30-8906895959ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.626925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:30.638269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-20980515-baad-4664-8b30-8906895959ca None None] MFA Rules not processed for user `8d87047620874e09bc5d851978949acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.639681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20980515-baad-4664-8b30-8906895959ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.642840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1216/2453] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:30.646758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.657196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680fb8-eb4c-4f0b-bbc8-04ceccb681b6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.661879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.662497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.662810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.663116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.663755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.665365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.675547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1217/2454] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:30.679674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.691664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32fd3a-666e-44ee-9eab-8419dfbd95cf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.695172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1238/2455] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:30.697246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.697841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.698123 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.698393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.699057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.700662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.707179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20980515-baad-4664-8b30-8906895959ca None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.708550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20980515-baad-4664-8b30-8906895959ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.712694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1218/2456] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:30.716696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.719888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20980515-baad-4664-8b30-8906895959ca None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.724033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.724627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.724902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.725156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.726506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d2d3-66ac-40e8-b318-76523a509cd6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.731248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.731899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.732218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.732460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] PATH_INFO: `/v3/projects/04acba854e5b4a01a2a24f4e29534fd8/users/100c5338120949ad9234f5f83e8efbcc/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.733844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.735512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25c0b0-0261-4776-b0c7-2a745fe92447 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.743738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1239/2457] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:30.747778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04acba854e5b4a01a2a24f4e29534fd8, user_id=100c5338120949ad9234f5f83e8efbcc, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.750449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16069ca4-bd41-4dea-a214-3cf9b112c8e6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:30.765282 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1219/2458] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:30 2022] PUT /identity/v3/projects/04acba854e5b4a01a2a24f4e29534fd8/users/100c5338120949ad9234f5f83e8efbcc/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:30.777408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.777605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.777941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.809388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] MFA Rules not processed for user `100c5338120949ad9234f5f83e8efbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.810640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.825755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:30.889102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1240/2459] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:30.892931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e565-6ba1-448a-81e6-56cf68ae2008 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.893323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e565-6ba1-448a-81e6-56cf68ae2008 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.893710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:30.894393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065e565-6ba1-448a-81e6-56cf68ae2008 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.896422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1220/2460] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 31 04:12:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 04:12:30.904043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.904567 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.905067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.928802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] MFA Rules not processed for user `ed122ea299374f9cb2b8b4854df14d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:30.930269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:30.966051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f23a6-ee6c-4961-8a6e-06e9e8e5f8c8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:30.970926 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:30.972001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:30.972001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:30.972291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:30.973002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:30.975081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.008137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1241/2461] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:31.012889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.025238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1221/2462] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 31 04:12:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:31.039840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.041729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:31.088812 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6aaf-41b7-4f8f-9b40-02953ee73879 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.093903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.094643 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.095053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.095683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.096162 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.098521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.112014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef710b8-1cae-49ba-b44e-be6991742b04 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.122533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1242/2463] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:31 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:31.126313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.136400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d7c6d-4bca-4554-95e3-326fa039b5be admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.141395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.142004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.142291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.142560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.143236 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.144851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.156027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1243/2464] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:31.161407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.168941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.169647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.169987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.170315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.173736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6299-a96e-43d6-a09b-df968d88cc5a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.180021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.180021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.180318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.180419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:31.180638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] PATH_INFO: `/v3/projects/c1343825f95f4422ad98647f0a37def1/users/d18a873f13bb46eb9687f2a9c524fd3a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.197332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1343825f95f4422ad98647f0a37def1, user_id=d18a873f13bb46eb9687f2a9c524fd3a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.200078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.203777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:31.222156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1244/2465] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:31 2022] PUT /identity/v3/projects/c1343825f95f4422ad98647f0a37def1/users/d18a873f13bb46eb9687f2a9c524fd3a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:31.226851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.250278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.252566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.268746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1222/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:12:31.312591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc2b90-965f-4a25-8c72-fc987d154afc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.321550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.322573 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.324558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.325012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.326097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.330242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450adc2-1182-422b-bfeb-093b2dca3072 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.349987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1245/2467] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:31.356193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.374300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf608f84-2b38-4d29-be2b-6c735fa9631f service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.381359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.382192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.382672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.383340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] PATH_INFO: `/v3/projects/c1343825f95f4422ad98647f0a37def1/users/d18a873f13bb46eb9687f2a9c524fd3a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.385576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1343825f95f4422ad98647f0a37def1, user_id=d18a873f13bb46eb9687f2a9c524fd3a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.389968 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.404217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1223/2468] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:31 2022] PUT /identity/v3/projects/c1343825f95f4422ad98647f0a37def1/users/d18a873f13bb46eb9687f2a9c524fd3a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:31.416496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.416886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.417260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.471349 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] MFA Rules not processed for user `d18a873f13bb46eb9687f2a9c524fd3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:31.472841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:31.571501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1246/2469] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:31.581176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.583055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:31.658317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b4207-d412-4f21-9f8c-4b6b9195b0f8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.662160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.663155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.663542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.663916 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.674145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.675771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0af93d-aaa6-47cf-af18-1735b642bf4d service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:31.686798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1224/2470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:31.869857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:31.952717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ab97b-725e-4cf0-ab09-6d833d83a41a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:31.961601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:31.963332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:31.963762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:31.964138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] PATH_INFO: `/v3/users/100c5338120949ad9234f5f83e8efbcc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:31.966010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=100c5338120949ad9234f5f83e8efbcc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:31.968400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:32.001482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:32.043580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100c5338120949ad9234f5f83e8efbcc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.059969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100c5338120949ad9234f5f83e8efbcc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.067434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100c5338120949ad9234f5f83e8efbcc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.068158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100c5338120949ad9234f5f83e8efbcc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.068990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100c5338120949ad9234f5f83e8efbcc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.086995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1247/2471] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:31 2022] DELETE /identity/v3/users/100c5338120949ad9234f5f83e8efbcc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:32.092390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:32.092995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:32.095615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:32.128088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:32.129343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:32.210079 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1225/2472] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:32.220256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:32.221700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:32.292366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a35e5-0c48-44b9-ad52-e7de2e5037aa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:32.296900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:32.297760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:32.298221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:32.298719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:32.310017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:32.311660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:32.324138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1248/2473] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:32.617034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:32.620322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:32.635725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201773-f34b-4f53-b09c-acd697afc0d0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:32.640642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:32.641321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:32.641603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:32.641874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] PATH_INFO: `/v3/projects/04acba854e5b4a01a2a24f4e29534fd8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:32.643431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04acba854e5b4a01a2a24f4e29534fd8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:32.645537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:32.673847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:32.698640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04acba854e5b4a01a2a24f4e29534fd8', 'request_id': 'req-0130412a-7724-4ee5-be0c-b2092410320a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.705449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04acba854e5b4a01a2a24f4e29534fd8', 'request_id': 'req-0130412a-7724-4ee5-be0c-b2092410320a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.712063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04acba854e5b4a01a2a24f4e29534fd8', 'request_id': 'req-0130412a-7724-4ee5-be0c-b2092410320a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.731985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] The token cache is being invalidate because project 04acba854e5b4a01a2a24f4e29534fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:32.732141 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.733249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.735099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1226/2474] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:32 2022] DELETE /identity/v3/projects/04acba854e5b4a01a2a24f4e29534fd8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:32.744131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:32.754962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d5830-3d1a-43ac-a7c2-e072b19d9f40 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:32.814454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:32.815238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:32.815582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:32.815918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] PATH_INFO: `/v3/users/d18a873f13bb46eb9687f2a9c524fd3a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:32.817233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18a873f13bb46eb9687f2a9c524fd3a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:32.819727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:32.876972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:32.883216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18a873f13bb46eb9687f2a9c524fd3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.896590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18a873f13bb46eb9687f2a9c524fd3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.904368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18a873f13bb46eb9687f2a9c524fd3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.905134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18a873f13bb46eb9687f2a9c524fd3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.905742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4764fdc7-846d-4587-ac4e-6f9d6d607392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18a873f13bb46eb9687f2a9c524fd3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:32.920878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1249/2475] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:32 2022] DELETE /identity/v3/users/d18a873f13bb46eb9687f2a9c524fd3a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:33.204766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.206208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:33.221167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130412a-7724-4ee5-be0c-b2092410320a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.288181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.288341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.288856 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.289438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] PATH_INFO: `/v3/projects/c1343825f95f4422ad98647f0a37def1` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.290799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1343825f95f4422ad98647f0a37def1)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.292867 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.345767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1343825f95f4422ad98647f0a37def1', 'request_id': 'req-d5a30f83-4cc0-45c4-817a-91b08d857bed'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:33.352290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1343825f95f4422ad98647f0a37def1', 'request_id': 'req-d5a30f83-4cc0-45c4-817a-91b08d857bed'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:33.358294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1343825f95f4422ad98647f0a37def1', 'request_id': 'req-d5a30f83-4cc0-45c4-817a-91b08d857bed'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:33.373708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] The token cache is being invalidate because project c1343825f95f4422ad98647f0a37def1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:33.374087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:33.374737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d5a30f83-4cc0-45c4-817a-91b08d857bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:33.376380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1227/2476] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:33 2022] DELETE /identity/v3/projects/c1343825f95f4422ad98647f0a37def1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:33.381635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.382392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.382392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.403075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:33.404348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:33.473289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1250/2477] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:33.483068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.483499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.483904 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.505100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:33.506430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:33.526823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1228/2478] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:33.528492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.539982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231a88d-8e93-40d1-8a5f-31d2ef6fb0e9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.545380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.546004 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.546441 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.546704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.547400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:12:33.547778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.549317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.567104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1251/2479] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:12:33.569125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.579139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fb6a4-2b59-4882-8625-ca163b085450 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.584325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.585386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.585725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.586057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.586779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.588785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.618689 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1229/2480] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:33 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:33.622871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.633438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763d4ff-b494-47e0-9fb1-1ceb79a1dcfc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.693330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.693977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.694255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.694738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.695449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.697436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.724422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1252/2481] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:33 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:33.725839 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.737491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d592a-a5c2-4e81-9883-920f1a5f5f26 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.743717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.744349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.744349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.744349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.744349 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.747231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.763737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1230/2482] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:33.764218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.775836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c48f6-3851-42ca-b42e-86864077209f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.781820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.782754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.782891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.783206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.783859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.785478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24343-8a8b-4330-9753-2fa0d3368fd4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.798112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1253/2483] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:33.802264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.807549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:33.814014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc118951-397d-47fa-ac94-11fd6a64e795 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:33.820706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:33.821495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.821875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.822407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] PATH_INFO: `/v3/projects/b4aef7b3f7b94522a89ff80497b63961/users/c9a7bdbad93d4c5ba17ea9e99bb2336b/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.839365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4aef7b3f7b94522a89ff80497b63961, user_id=c9a7bdbad93d4c5ba17ea9e99bb2336b, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:33.842625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:33.858625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1231/2484] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:33 2022] PUT /identity/v3/projects/b4aef7b3f7b94522a89ff80497b63961/users/c9a7bdbad93d4c5ba17ea9e99bb2336b/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:33.870145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:33.870427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:33.870717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:33.900273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] MFA Rules not processed for user `c9a7bdbad93d4c5ba17ea9e99bb2336b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:33.901392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:33.914529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:33.974046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1254/2485] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:33.986107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:33.987994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:34.046964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56594-fba5-43a6-bbd8-99cda4de2f8c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:34.050772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:34.052392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.052519 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.052751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.062470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:34.064016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a41b0b-4597-4f6f-af39-50a284750bcc service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:34.073711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1232/2486] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:34.094752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:34.161270 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8d26f-0b28-4c60-968e-8db9f158141a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:34.166034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:34.166668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.166949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.167265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] PATH_INFO: `/v3/users/c9a7bdbad93d4c5ba17ea9e99bb2336b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.168437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a7bdbad93d4c5ba17ea9e99bb2336b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:34.170389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:34.220446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a7bdbad93d4c5ba17ea9e99bb2336b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.233028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a7bdbad93d4c5ba17ea9e99bb2336b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.240254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a7bdbad93d4c5ba17ea9e99bb2336b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.241044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a7bdbad93d4c5ba17ea9e99bb2336b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.242049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a7bdbad93d4c5ba17ea9e99bb2336b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.255678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1255/2487] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:34 2022] DELETE /identity/v3/users/c9a7bdbad93d4c5ba17ea9e99bb2336b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:34.260380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.260882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.260981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.285069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:34.286271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:34.345281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1233/2488] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:34.355271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:34.356796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:34.394282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:34.417719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f4d1e-4189-4280-bdcf-a5a5679ac133 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:34.421841 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:34.422444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.422718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.422987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.432402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:34.433843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8f79-100c-44f7-bacf-5bb46ab68b51 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:34.443460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1256/2489] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:34.834666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:34.836258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:34.859144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bdeab-5f71-4965-9101-6bfa487d4676 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:34.868651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:34.869556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.869821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.870000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] PATH_INFO: `/v3/projects/b4aef7b3f7b94522a89ff80497b63961` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.871879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4aef7b3f7b94522a89ff80497b63961)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:34.874618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:34.927144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4aef7b3f7b94522a89ff80497b63961', 'request_id': 'req-18184062-9bfb-480a-a889-9a258b9e78a1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.933485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4aef7b3f7b94522a89ff80497b63961', 'request_id': 'req-18184062-9bfb-480a-a889-9a258b9e78a1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.939108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4aef7b3f7b94522a89ff80497b63961', 'request_id': 'req-18184062-9bfb-480a-a889-9a258b9e78a1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.952910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] The token cache is being invalidate because project b4aef7b3f7b94522a89ff80497b63961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:34.953197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.953825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-18184062-9bfb-480a-a889-9a258b9e78a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:34.958877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1234/2490] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:34 2022] DELETE /identity/v3/projects/b4aef7b3f7b94522a89ff80497b63961 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:34.960816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:34.961102 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:34.961374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:34.981605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:34.983388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:35.045380 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1257/2491] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:12:35.053534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.054286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.054286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.062943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:35.075431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:35.077027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:35.085983 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1235/2492] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:35.090298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.100173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08363495-d34a-46f1-af7a-08bd51c1e15d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.105543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.106146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.106424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.106694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.107382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:12:35.108020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.109635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.122435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1258/2493] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:12:35.127099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.136745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c457df6-6692-46e3-846e-91bc5e5b45c8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.141554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.142290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.142594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.142862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.143546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.145495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570af136-dabc-4e71-abc9-46801dd199b6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.183815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1236/2494] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:35.188027 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.200212 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1dd1-52ab-42c1-93b1-0e8322d8626c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.235065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.235491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.235860 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.256446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:35.257942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:35.287142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.288955 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.289247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.289519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.290178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.292213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.322866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1259/2495] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:35.327372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.339957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1237/2496] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:35.351265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562016-2b01-4270-9b1a-9005a53076e6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.353092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.354475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:35.356699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.357565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.358297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.358671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.359448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.361319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.372884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1260/2497] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:35.376907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.390457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bbc8b-3417-4f2a-949b-483db68cf033 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.395528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.396133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.396395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.396670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.397634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.399261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.410133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1261/2498] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:35.414705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.425804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcb702-c6cb-4d1e-ac40-32cb037a3dc0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.431477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.432094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.432523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.432795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] PATH_INFO: `/v3/projects/0daf9a95365842abbfb0f3befbb7de03/users/3701e1df7036495a81bf0292cf54cedb/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.436340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde98731-bb0e-4e83-9174-ce43525b2d79 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.441758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.442493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.442774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.443749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.452837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.453043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0daf9a95365842abbfb0f3befbb7de03, user_id=3701e1df7036495a81bf0292cf54cedb, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.454458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9418bb-06f8-4176-b913-9e039c7a673a service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.456088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.463820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1238/2499] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 04:12:35.477916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1262/2500] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:35 2022] PUT /identity/v3/projects/0daf9a95365842abbfb0f3befbb7de03/users/3701e1df7036495a81bf0292cf54cedb/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:12:35.499789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.500258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.500724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.532303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] MFA Rules not processed for user `3701e1df7036495a81bf0292cf54cedb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:35.534273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:35.630518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:35.637447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1239/2501] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:35.642460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.654142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:35.736302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7f3d-ca96-4e81-ae74-c23debbbd403 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.742264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.742910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.743226 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.743618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.744373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.746520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.787098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1263/2502] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:12:35.789295 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.807804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d075c-01b7-4a98-abd8-f15a9b88ea54 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.878228 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.879165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.879546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.879829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.880475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.884804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.910401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1240/2503] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:35 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:35.915333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.927493 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892981b-ddf3-47bb-add2-1a963c7fc87d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.932629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.933253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.933532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.933804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.934455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.936648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.958714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:35.965830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1264/2504] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:35.970237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4addbfc-999b-4d33-ae0f-513d9bcef934 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:35.975774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:35.976717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:35.976967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:35.977303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:35.978024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:35.979946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d6af8-74dd-4c98-88ef-43f565eb8298 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:35.997918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.005254 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1241/2505] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:36.014124 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c25ed2-4224-4c57-9e27-534eadd0b39c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.022440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.023350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.025543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.025983 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] PATH_INFO: `/v3/projects/59c511bf39864d0b8c20efca431c8314/users/f11016823a41452eb6b5618177aacf3d/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.055655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c511bf39864d0b8c20efca431c8314, user_id=f11016823a41452eb6b5618177aacf3d, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.059708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.087569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1265/2506] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:35 2022] PUT /identity/v3/projects/59c511bf39864d0b8c20efca431c8314/users/f11016823a41452eb6b5618177aacf3d/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:36.095536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.096627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.109796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.169817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] MFA Rules not processed for user `f11016823a41452eb6b5618177aacf3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:36.173815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:36.300797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1242/2507] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:36.306021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.395543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ffdff-cc06-49d3-b94a-c673dc59f346 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.401898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.402517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.402794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.403086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.403726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.405724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.455077 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1266/2508] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:36 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:36.459702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.472944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482687-0eaa-4b92-bf0f-2e4115329e09 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.616069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.617064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.617464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.617800 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.618545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.620670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.650413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:36.662820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1243/2509] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:36 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:36.664024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.680965 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07e110-002e-4f70-b312-1f6e577427eb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.694445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.695219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.695500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.695858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.696519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.698350 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.723768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1267/2510] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:36.725182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.738834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b586a49-b071-4c32-9d67-b7b3c96754d6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.744971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.745745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.746001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.746277 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] PATH_INFO: `/v3/projects/437a7960ed7e4e629057f9c5b532a4db/users/c33c3f05566d4ab59053765a0144868b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.766569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437a7960ed7e4e629057f9c5b532a4db, user_id=c33c3f05566d4ab59053765a0144868b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.769347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.798116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1244/2511] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:36 2022] PUT /identity/v3/projects/437a7960ed7e4e629057f9c5b532a4db/users/c33c3f05566d4ab59053765a0144868b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:36.799249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.810296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9aae5-b7ab-4c18-9f90-097f61cfc3f9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.854885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:36.890225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.890942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.891251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.891520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.892185 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.894135 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b927c-296c-4213-b222-e55ffa0dcba8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.914991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1268/2512] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:12:36.919088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:36.932250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9f24b-1611-4224-a69f-44cc39bbabf1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:36.939716 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:36.940393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.940669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.940944 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] PATH_INFO: `/v3/projects/437a7960ed7e4e629057f9c5b532a4db/users/c33c3f05566d4ab59053765a0144868b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:36.942872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=437a7960ed7e4e629057f9c5b532a4db, user_id=c33c3f05566d4ab59053765a0144868b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:36.945893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:36.957987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1245/2513] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:36 2022] PUT /identity/v3/projects/437a7960ed7e4e629057f9c5b532a4db/users/c33c3f05566d4ab59053765a0144868b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:36.970692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:36.970985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:36.971272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:37.004594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] MFA Rules not processed for user `c33c3f05566d4ab59053765a0144868b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:37.006346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:37.088331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1269/2514] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:37.099535 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:37.101264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:37.163560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7b0f5-6bfc-45fb-b537-353de0bd6fde admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:37.167401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:37.168018 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:37.168297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:37.168563 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:37.236473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:37.237935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:37.249269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1246/2515] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:38.854434 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:38.856026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:38.873053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65feb1f-4112-4b6c-9c8b-f4e1c92fc872 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:38.885635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:38.886274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:38.886562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:38.886834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:38.996391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:38.997951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:39.010743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1270/2516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:39.707602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:39.716804 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5eac-87e1-4104-a98d-181218dbc7a4 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:39.720513 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:39.721156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:39.721435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:39.721842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:39.731786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:39.733636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9569bb7-9704-48b9-9152-7abd7531bf62 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:39.744025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1247/2517] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:43.952713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:44.006663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:44.020406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee896785-692b-4c50-a282-ca2545499f01 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:44.026039 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:44.026711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:44.026977 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:44.027285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] PATH_INFO: `/v3/users/3701e1df7036495a81bf0292cf54cedb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:44.028458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3701e1df7036495a81bf0292cf54cedb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:44.030409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:44.083554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3701e1df7036495a81bf0292cf54cedb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.095111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3701e1df7036495a81bf0292cf54cedb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.102268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3701e1df7036495a81bf0292cf54cedb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.103020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3701e1df7036495a81bf0292cf54cedb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.103735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3701e1df7036495a81bf0292cf54cedb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.119265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1271/2518] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:43 2022] DELETE /identity/v3/users/3701e1df7036495a81bf0292cf54cedb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:44.123881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:44.124419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:44.124419 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:44.144565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:44.145808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:44.218404 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1248/2519] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:44.228544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:44.230030 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:44.291891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bef48f-68e2-4d7c-aa4d-0b02a093aa95 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:44.296292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:44.296901 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:44.297174 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:44.297437 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:44.307244 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:44.308680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:44.318369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1272/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:44.848179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:44.849687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:44.857895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:44.866221 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749db7fc-e442-4dc0-8a99-5b9fdc067ff7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:44.872898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:44.873806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:44.874082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:44.874356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] PATH_INFO: `/v3/projects/0daf9a95365842abbfb0f3befbb7de03` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:44.875619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0daf9a95365842abbfb0f3befbb7de03)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:44.877677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:44.932436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0daf9a95365842abbfb0f3befbb7de03', 'request_id': 'req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.939014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0daf9a95365842abbfb0f3befbb7de03', 'request_id': 'req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.944782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0daf9a95365842abbfb0f3befbb7de03', 'request_id': 'req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.959014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] The token cache is being invalidate because project 0daf9a95365842abbfb0f3befbb7de03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:44.959422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.960092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:44.961974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1249/2521] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:44 2022] DELETE /identity/v3/projects/0daf9a95365842abbfb0f3befbb7de03 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:44.966002 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:44.976903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce26b9-7194-49dd-8d93-31edbb6c497b service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:45.035953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:45.036599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:45.036881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:45.037155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] PATH_INFO: `/v3/users/f11016823a41452eb6b5618177aacf3d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:45.038287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11016823a41452eb6b5618177aacf3d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:45.040623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:45.057724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:45.092962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11016823a41452eb6b5618177aacf3d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.106252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11016823a41452eb6b5618177aacf3d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.113368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11016823a41452eb6b5618177aacf3d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.114080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11016823a41452eb6b5618177aacf3d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.114692 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11016823a41452eb6b5618177aacf3d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.129704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1273/2522] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:44 2022] DELETE /identity/v3/users/f11016823a41452eb6b5618177aacf3d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:45.412010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:45.413010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:45.445137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6dbf5a-dd2f-4994-96cc-1a9871e61504 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:45.508779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:45.509467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:45.509741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:45.510012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] PATH_INFO: `/v3/projects/59c511bf39864d0b8c20efca431c8314` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:45.511156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c511bf39864d0b8c20efca431c8314)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:45.513080 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:45.562830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c511bf39864d0b8c20efca431c8314', 'request_id': 'req-fe5b643d-7904-4942-93d2-f292e995cb51'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.569288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c511bf39864d0b8c20efca431c8314', 'request_id': 'req-fe5b643d-7904-4942-93d2-f292e995cb51'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.575732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c511bf39864d0b8c20efca431c8314', 'request_id': 'req-fe5b643d-7904-4942-93d2-f292e995cb51'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.591216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] The token cache is being invalidate because project 59c511bf39864d0b8c20efca431c8314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:45.591507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.592143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.593667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1250/2523] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:45 2022] DELETE /identity/v3/projects/59c511bf39864d0b8c20efca431c8314 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:45.597963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:45.599542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:45.613679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ef985-c56a-402a-bbaa-6a05fb99c39b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:45.672743 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:45.673373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:45.673608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:45.673907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] PATH_INFO: `/v3/users/c33c3f05566d4ab59053765a0144868b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:45.675060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33c3f05566d4ab59053765a0144868b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:45.677011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:45.729476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33c3f05566d4ab59053765a0144868b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.742121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33c3f05566d4ab59053765a0144868b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.749658 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33c3f05566d4ab59053765a0144868b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.750455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33c3f05566d4ab59053765a0144868b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.751232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6105a4a3-e0a6-4e3d-98a2-3b0296a22ad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33c3f05566d4ab59053765a0144868b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:45.766608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1274/2524] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:12:45 2022] DELETE /identity/v3/users/c33c3f05566d4ab59053765a0144868b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:46.060598 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.064728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:46.081178 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b643d-7904-4942-93d2-f292e995cb51 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.098167 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:46.158051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.158844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.159265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.159629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] PATH_INFO: `/v3/projects/437a7960ed7e4e629057f9c5b532a4db` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.160865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=437a7960ed7e4e629057f9c5b532a4db)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.162910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.226599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437a7960ed7e4e629057f9c5b532a4db', 'request_id': 'req-692bcd65-338d-4db5-92f9-ac2bd67188cc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:46.243348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437a7960ed7e4e629057f9c5b532a4db', 'request_id': 'req-692bcd65-338d-4db5-92f9-ac2bd67188cc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:46.243348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437a7960ed7e4e629057f9c5b532a4db', 'request_id': 'req-692bcd65-338d-4db5-92f9-ac2bd67188cc'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:46.254416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] The token cache is being invalidate because project 437a7960ed7e4e629057f9c5b532a4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:12:46.254734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:46.255383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-692bcd65-338d-4db5-92f9-ac2bd67188cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:12:46.260957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1251/2525] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:12:46 2022] DELETE /identity/v3/projects/437a7960ed7e4e629057f9c5b532a4db => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:46.262860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.264116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.269939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.297540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:46.298796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:46.304814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:46.372673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1275/2526] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:46.381523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.381846 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.382233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.405056 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:46.405795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:46.414951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1252/2527] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:46.419283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.429768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eedeaf-f1ea-4f91-b04c-1a2c26bd3579 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.434997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.435740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.436112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.436508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.437501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:12:46.437993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.441318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.467136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1276/2528] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:12:46.469344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.480050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7dc7a-8bd7-48ba-acc8-14ae1a1b905c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.485914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.486595 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.486932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.487300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.487996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.490351 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.526943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1253/2529] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:12:46 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:46.528181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.539319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391ecde-155e-4752-a8d7-748b6d814198 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.600299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.601011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.601358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.601695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.602499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.604530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.628346 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1277/2530] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:12:46 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:12:46.632781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.645122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055a294-5cce-4026-82c9-cbf137a4e69a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.649842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.650529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.650808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.651115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.651812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.653642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.664504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1254/2531] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:46.668781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.679510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe23d-2b5d-4f2a-870c-574227068649 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.685911 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.686546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.686845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.687139 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.687783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.689403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488b72d-a424-447a-beda-247079583ec9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.701396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1278/2532] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:12:46.705600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.716161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853cd3f-8f91-4f15-8fba-a74aee927d75 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.721003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.721740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.722043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.722313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] PATH_INFO: `/v3/projects/175f95fba7384b58b9c75ea11c6e3469/users/06b6013a9d5044d1bb2d033a95c55a09/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.738743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=175f95fba7384b58b9c75ea11c6e3469, user_id=06b6013a9d5044d1bb2d033a95c55a09, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.741430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.756879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1255/2533] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:12:46 2022] PUT /identity/v3/projects/175f95fba7384b58b9c75ea11c6e3469/users/06b6013a9d5044d1bb2d033a95c55a09/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:12:46.762746 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.763055 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.763300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.783084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:46.784249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:46.850564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1279/2534] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:46.860820 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:46.862853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:46.911746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:46.926310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723ff8-d03e-475e-ba5d-e3f15e5772dd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:46.931120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:46.931742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:46.932051 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:46.932323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:46.946869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:46.948628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7132-e20c-4307-9999-1d2ad5306a56 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:46.958914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1256/2535] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:47.540263 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:47.542664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:47.587659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:12:47.675838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa669065-74e5-4fed-afad-9a22a421ff24 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:47.688651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:47.689458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:47.689836 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:47.690214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:47.692199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:47.705936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:47.707794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:47.720462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1280/2536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:47.949077 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a521b23-ce15-4dfd-8a28-7f98d90e8467 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:47.949641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a521b23-ce15-4dfd-8a28-7f98d90e8467 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:47.949641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a521b23-ce15-4dfd-8a28-7f98d90e8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:47.985206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0a521b23-ce15-4dfd-8a28-7f98d90e8467 None None] MFA Rules not processed for user `06b6013a9d5044d1bb2d033a95c55a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:47.986597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0a521b23-ce15-4dfd-8a28-7f98d90e8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:48.062978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1257/2537] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:48.074972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:48.076609 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:48.169308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaebe50-9919-4b74-b48e-bdfb16625359 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:48.173780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:48.174393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:48.174675 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:48.174940 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:48.186379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:48.188408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26107cff-1913-4e8b-8b67-399dcd986668 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:48.198967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1281/2538] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:12:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:12:49.600286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:49.600816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:49.600816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:49.624088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] MFA Rules not processed for user `06b6013a9d5044d1bb2d033a95c55a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:49.625902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:49.636837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1258/2539] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:12:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:49.888357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b55e0-a63f-48e5-8444-a2ec1a260300 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:49.888839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b55e0-a63f-48e5-8444-a2ec1a260300 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:49.888949 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b55e0-a63f-48e5-8444-a2ec1a260300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:49.908754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8b4b55e0-a63f-48e5-8444-a2ec1a260300 None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:12:49.909961 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8b4b55e0-a63f-48e5-8444-a2ec1a260300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:49.964025 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1282/2540] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:12:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:12:49.986981 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:12:49.988445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:12:50.062023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0e3a0-5f04-4e75-90ae-ae8baadc9fa0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:12:50.067208 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:12:50.067819 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:12:50.068148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:12:50.068428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:12:50.077893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:12:50.079606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd088-3e75-4d3f-9147-ecb455e4a9df service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:12:50.088797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1259/2541] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:12:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:12:59.957329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=77 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:00.085200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=74 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:05.128385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:05.128887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:05.128982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:05.149188 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] MFA Rules not processed for user `06b6013a9d5044d1bb2d033a95c55a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:05.150398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:05.158571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1283/2542] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:15.143234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:24.027910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:24.028450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:24.028556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:24.048410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:24.049600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:24.058245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1260/2543] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 31 04:13:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:24.071542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:24.084787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62abf-8f99-41b9-91d0-e21e93505344 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:24.090796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:24.091481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:24.091757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:24.092023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:24.100372 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:24.101966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:24.110413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1284/2544] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:13:32.549523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:32.551411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:32.563989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbac6b-2e96-4786-8ad9-273f389455ff None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:32.567801 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:32.568439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:32.568720 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:32.568994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:32.570941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:32.586763 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:32.589154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:32.601307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1261/2545] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:34.106898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:37.117943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:37.119602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:37.242859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:37.245059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:37.269334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a2441-c2f8-4fbe-875a-067a4d4d1b69 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:37.282156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:37.284324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:37.284850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:37.285288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] PATH_INFO: `/v3/users/956b08da3ba944d382877cf266389127` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:37.286978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] RBAC: Authorizing `identity:delete_user(user_id=956b08da3ba944d382877cf266389127)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:37.291831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ed88e-7c98-497d-8f71-7962c32d3e6b service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:37.292607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:37.310168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:37.315897 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:37.316994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:37.318623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:37.325171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:37.355059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:37.366788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:37.381270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956b08da3ba944d382877cf266389127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.382000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956b08da3ba944d382877cf266389127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.395960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1285/2546] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:37.396489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956b08da3ba944d382877cf266389127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.416996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956b08da3ba944d382877cf266389127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.417631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7e7d1498-f1a8-4223-8133-135a1ad1548f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956b08da3ba944d382877cf266389127'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.459156 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1262/2547] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:37 2022] DELETE /identity/v3/users/956b08da3ba944d382877cf266389127 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:37.656496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:37.678360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d06cb-3e76-4ef1-afdb-4c0374c57702 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:37.760520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:37.761440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:37.761838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:37.762206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] PATH_INFO: `/v3/projects/66bcbde4eda6417ea15dca2371c28540` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:37.763483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bcbde4eda6417ea15dca2371c28540)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:37.766463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:37.861636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bcbde4eda6417ea15dca2371c28540', 'request_id': 'req-384e4a26-4035-4ba8-863b-fd34b5335194'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.870696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bcbde4eda6417ea15dca2371c28540', 'request_id': 'req-384e4a26-4035-4ba8-863b-fd34b5335194'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.883301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bcbde4eda6417ea15dca2371c28540', 'request_id': 'req-384e4a26-4035-4ba8-863b-fd34b5335194'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.903550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] The token cache is being invalidate because project 66bcbde4eda6417ea15dca2371c28540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:37.904014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.904763 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-384e4a26-4035-4ba8-863b-fd34b5335194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:37.907175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1286/2548] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:37 2022] DELETE /identity/v3/projects/66bcbde4eda6417ea15dca2371c28540 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:37.911883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:37.912173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:37.912448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:37.942114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:37.942730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:38.060664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1263/2549] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:38.068960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.069322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.069645 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.105086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:38.107737 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:38.117245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1287/2550] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:38.121160 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.144515 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584044-eca9-4f6b-835b-40daa0056aab None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.151931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.152546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.152823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.153092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.153746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:13:38.155040 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.157211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.189496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1264/2551] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:13:38.194032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.205632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01999df-92c5-47cb-9e33-efb19781ec1a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.211330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.212344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.213061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.213555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.215239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.217391 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.235059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:38.261054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1288/2552] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:38 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:13:38.265146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.278360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2cc2-d7ef-4d22-8032-7d10f794107b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.347367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:38.399406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.400408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.401007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.402245 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.403174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.405702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.462032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1265/2553] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:38 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:38.466618 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.481880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7075f7c-ea82-4893-a5d5-9908ef88cd71 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.500240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.500956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.501465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.501906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.502777 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.504780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.533738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1289/2554] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:38.538307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.551456 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c13f-74ce-4ebc-ad20-55e0f02c9115 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.566361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.567041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.567398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.567780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.568522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.573368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70028d-8642-4762-badb-151734fa99eb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.590511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1266/2555] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:38.591294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.607748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45de07c-584d-4ff0-acc9-aa668306b8a5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.621063 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.621938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.622410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.622851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] PATH_INFO: `/v3/projects/b2f251186d5346d9ae213f018e299cb4/users/e29d3d1942614652b3229888d04dc5bf/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.652042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f251186d5346d9ae213f018e299cb4, user_id=e29d3d1942614652b3229888d04dc5bf, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.659449 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.682204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1290/2556] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:38 2022] PUT /identity/v3/projects/b2f251186d5346d9ae213f018e299cb4/users/e29d3d1942614652b3229888d04dc5bf/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:13:38.684296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4470f30-165f-4dc1-a840-c3ead4085be1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.685084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4470f30-165f-4dc1-a840-c3ead4085be1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.685492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4470f30-165f-4dc1-a840-c3ead4085be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.716315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d4470f30-165f-4dc1-a840-c3ead4085be1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:38.717737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d4470f30-165f-4dc1-a840-c3ead4085be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:38.821821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1267/2557] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:13:38.831931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:38.833982 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:38.932555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78dfbfb-902e-4d6f-aea4-635f3a9c7b20 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:38.937828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:38.938580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:38.939142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:38.939528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:38.959718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:38.962028 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:38.978311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1291/2558] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:43.087938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:43.088833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:43.089275 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:43.111410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] MFA Rules not processed for user `5eeb4f0ca36d40e4ad8d5b77cdf35068`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:43.112775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:43.205220 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1268/2559] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 31 04:13:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:43.223660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:43.226001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:43.320425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33be998-b999-4c30-9f8f-bbeb0a32c09b service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:43.327065 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b526cee1dc234083a7f4897fee8e22b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:43.328045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:43.328388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:43.328734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:43.338547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:43.340297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b09af-9a39-4fad-ab42-06a372eebd83 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:43.371538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1292/2560] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 31 04:13:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 04:13:44.226117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:44.228553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:44.242352 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e10d73-4c3a-4e69-9914-3028a86338bc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:44.247636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:44.248635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:44.248938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:44.249238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:44.251195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:44.266792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:44.268330 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:44.280177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1269/2561] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:45.563839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:45.564365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:45.564365 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:45.639798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] MFA Rules not processed for user `e29d3d1942614652b3229888d04dc5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:45.641044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:45.820457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:45.896992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:45.899009 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:45.907013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1293/2562] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:45.995957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:46.046259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaedce-a7bb-43f7-acd1-b2aa1bf3dbdb service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:46.052592 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:46.053339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:46.053881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:46.054379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:46.064378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:46.067001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:46.081183 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1270/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:46.408091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:46.410424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:46.433544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf7336-0f99-4797-be98-9f59d87d6157 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:46.448720 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:46.449536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:46.449912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:46.450278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] PATH_INFO: `/v3/users/06b6013a9d5044d1bb2d033a95c55a09` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:46.456666 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b6013a9d5044d1bb2d033a95c55a09)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:46.459539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:46.533154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b6013a9d5044d1bb2d033a95c55a09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:46.567277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b6013a9d5044d1bb2d033a95c55a09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:46.577525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b6013a9d5044d1bb2d033a95c55a09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:46.578716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b6013a9d5044d1bb2d033a95c55a09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:46.580593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cf8ebdf8-b11d-46f4-a026-a7069ecf2376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b6013a9d5044d1bb2d033a95c55a09'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:46.613171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1294/2564] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:46 2022] DELETE /identity/v3/users/06b6013a9d5044d1bb2d033a95c55a09 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:46.994033 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:47.010787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c98d69-f2e3-4ab2-ad01-41e09a35e064 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:47.147113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:47.148176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.149134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.149708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] PATH_INFO: `/v3/projects/175f95fba7384b58b9c75ea11c6e3469` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.151314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=175f95fba7384b58b9c75ea11c6e3469)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:47.154616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:47.224894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175f95fba7384b58b9c75ea11c6e3469', 'request_id': 'req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:47.235276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175f95fba7384b58b9c75ea11c6e3469', 'request_id': 'req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:47.244680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175f95fba7384b58b9c75ea11c6e3469', 'request_id': 'req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:47.278673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] The token cache is being invalidate because project 175f95fba7384b58b9c75ea11c6e3469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:47.279472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:47.280437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-603db3f2-ba76-4d9e-b4d3-c3c5f62b8caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:47.287762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1271/2565] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:46 2022] DELETE /identity/v3/projects/175f95fba7384b58b9c75ea11c6e3469 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:47.292705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3a0db-4446-4313-b0e8-d2e92f42937d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.292991 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3a0db-4446-4313-b0e8-d2e92f42937d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.293259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3a0db-4446-4313-b0e8-d2e92f42937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.322471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-97d3a0db-4446-4313-b0e8-d2e92f42937d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:47.324257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-97d3a0db-4446-4313-b0e8-d2e92f42937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:47.402274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.403762 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.404235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.448530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1295/2566] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:47.456010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.456694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.457206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.479132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] MFA Rules not processed for user `62f5d4659eea4d91a689d709460396d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:47.488597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:47.502308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:47.506306 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:47.542371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1296/2567] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:13:47.547710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:47.566528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3f794-943e-4870-a8fd-6a5f6e2d73f2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:47.586086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:47.587287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.587458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.588231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.589107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:13:47.589563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:47.591395 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:47.626631 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1297/2568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:13:47.631006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:47.650767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1272/2569] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:47.653085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870bcb0-b5cd-4078-a22f-16594147243e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:47.665822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:47.666884 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.667417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.667929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.668809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:47.671207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:47.716029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1298/2570] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:47 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:47.720339 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:47.740806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26858d-2e0e-49df-b295-9c54d0066536 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:47.759830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:47.862747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:47.863499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.863777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.864096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.866375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:47.870362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:47.922267 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1273/2571] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:47 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:47.923149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:47.943960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b233a-abea-4251-a52b-a4598642c6e2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:47.957294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:47.958617 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:47.958898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:47.961178 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:47.963406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:47.965477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:47.975303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:47.990447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1299/2572] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:13:47.995898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:48.007923 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a4fe5-8286-4f6d-969a-8a7adc55b9c9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:48.023193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:48.023829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.024106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.024431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.025266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:48.034315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ad555-dc36-4ee4-81f7-273971c5e838 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:48.072290 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1274/2573] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:48.073428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:48.088464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47637275-87b1-4ce9-a62b-61a4be093eaa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:48.099058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:48.099855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.100209 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.100594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] PATH_INFO: `/v3/projects/0328f51d8c6544d38856a932f67d79f6/users/2fdbe4a487c84500858c7310b4ec9d3a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.138715 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0328f51d8c6544d38856a932f67d79f6, user_id=2fdbe4a487c84500858c7310b4ec9d3a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:48.141865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:48.175126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1300/2574] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:48 2022] PUT /identity/v3/projects/0328f51d8c6544d38856a932f67d79f6/users/2fdbe4a487c84500858c7310b4ec9d3a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:48.178566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.178980 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.179379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.236530 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] MFA Rules not processed for user `2fdbe4a487c84500858c7310b4ec9d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:48.238040 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:48.335903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1275/2575] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:48.348794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:48.350779 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:48.429448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfae33-0a14-4961-8d16-d09e0e70218d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:48.434570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:48.435243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.435520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.436234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.447858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:48.449313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e393-84b4-4a0c-a6e3-3a4465e325b0 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:48.459855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1301/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:48.500651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:48.682597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac6fd-0f86-4c3e-be33-4b87fe4261c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:48.690215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:48.691332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.691908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.692398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] PATH_INFO: `/v3/users/2fdbe4a487c84500858c7310b4ec9d3a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.705864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdbe4a487c84500858c7310b4ec9d3a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:48.708105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:48.753901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:48.799229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdbe4a487c84500858c7310b4ec9d3a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:48.800152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdbe4a487c84500858c7310b4ec9d3a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:48.811587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdbe4a487c84500858c7310b4ec9d3a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:48.824583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdbe4a487c84500858c7310b4ec9d3a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:48.825584 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdbe4a487c84500858c7310b4ec9d3a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:48.857002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1276/2577] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:48 2022] DELETE /identity/v3/users/2fdbe4a487c84500858c7310b4ec9d3a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:48.859016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c763-bc6c-464d-8f33-5c73cd15220f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:48.859594 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c763-bc6c-464d-8f33-5c73cd15220f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:48.860544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c763-bc6c-464d-8f33-5c73cd15220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:48.885406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4844c763-bc6c-464d-8f33-5c73cd15220f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:48.886592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4844c763-bc6c-464d-8f33-5c73cd15220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:48.958724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1302/2578] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:48.970828 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:48.972607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:49.082626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcb98a-44ac-44a0-8274-e9d743c7908a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:49.087092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:49.087813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:49.088239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:49.088996 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:49.113740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:49.115614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:49.140784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1277/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:49.217497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cddf-ba8a-4504-a6ad-f3034e186ae5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:49.218823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cddf-ba8a-4504-a6ad-f3034e186ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:49.219227 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45cddf-ba8a-4504-a6ad-f3034e186ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:49.260707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-db45cddf-ba8a-4504-a6ad-f3034e186ae5 None None] MFA Rules not processed for user `e29d3d1942614652b3229888d04dc5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:49.263245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-db45cddf-ba8a-4504-a6ad-f3034e186ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:49.406689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1303/2580] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:49.822864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:49.825505 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:49.848365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18292a7-5f4c-4f9e-806d-5448ade5707b service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:49.873408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:49.874333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:49.874778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:49.875240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] PATH_INFO: `/v3/projects/0328f51d8c6544d38856a932f67d79f6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:49.876918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0328f51d8c6544d38856a932f67d79f6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:49.879101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:49.999892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0328f51d8c6544d38856a932f67d79f6', 'request_id': 'req-59d16429-2086-40cf-94f5-9b1c19c28f07'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.019371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0328f51d8c6544d38856a932f67d79f6', 'request_id': 'req-59d16429-2086-40cf-94f5-9b1c19c28f07'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.031536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0328f51d8c6544d38856a932f67d79f6', 'request_id': 'req-59d16429-2086-40cf-94f5-9b1c19c28f07'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.053132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] The token cache is being invalidate because project 0328f51d8c6544d38856a932f67d79f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:50.053966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.054998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-59d16429-2086-40cf-94f5-9b1c19c28f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.057811 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1278/2581] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:49 2022] DELETE /identity/v3/projects/0328f51d8c6544d38856a932f67d79f6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:50.068232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.068761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.070107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.103870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:50.105431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.132706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:50.205154 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1304/2582] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:50.213459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.214393 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.214835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.246407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:50.247798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.257525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1279/2583] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:50.261943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.275842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca766297-fc98-412d-b6d1-5d5cd45dc9ad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.287097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.287695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.288513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.288513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.288890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:13:50.289301 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.291652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.311830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1305/2584] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:13:50.318553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.330179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694c8d0-4e24-404c-94d9-834221198fcb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.335826 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.336544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.337096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.338088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.339053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.341364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.371250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:50.392479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1280/2585] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:50 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:50.397291 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.410804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed810a5b-4da9-4310-8071-8ab4736fbf03 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.485697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.489398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.491830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.492807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.493356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.493817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.495409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.501903 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.538642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1306/2586] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:50 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:50.542939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.560725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eaaf34-3159-4ee6-b762-9c76ec6f8805 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.571896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.572849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.573333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.574054 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.575093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.576990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.616472 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1307/2587] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:13:50.617367 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.623550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.642741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e1564-1e8e-4423-bc5f-6a88aacf1655 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.647891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.648709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.649043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.649580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] PATH_INFO: `/v3/users/f80c8c1d6af7450eb8f936df231ad63b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.650752 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80c8c1d6af7450eb8f936df231ad63b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.651859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49940ecf-57a0-4721-a819-a20ebc94904f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.656062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.659259 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.659963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.660321 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.661269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.663250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.677591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.679781 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.693869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1281/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:13:50.695054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.707467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a745b38-ca9f-4a27-ac9d-3507dbe6613b service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.714314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.714619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.715071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.715405 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.716190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.718112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.730544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1282/2589] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:50.735555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.745427 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80c8c1d6af7450eb8f936df231ad63b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.745803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a73a57-1e39-4226-a613-2752fbfeb190 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:50.750672 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:50.751372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.751747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.752120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] PATH_INFO: `/v3/projects/492e20665a634eb3b48c6f3dadc5b57f/users/f5d8077b412241cd908ba839258c5024/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.760256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80c8c1d6af7450eb8f936df231ad63b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.769588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80c8c1d6af7450eb8f936df231ad63b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.769892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=492e20665a634eb3b48c6f3dadc5b57f, user_id=f5d8077b412241cd908ba839258c5024, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:50.770639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80c8c1d6af7450eb8f936df231ad63b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.771323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c35fa1f7-52a4-46eb-b77d-4f56d418a281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80c8c1d6af7450eb8f936df231ad63b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:50.773094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:50.785199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:50.792134 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1308/2590] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:50 2022] DELETE /identity/v3/users/f80c8c1d6af7450eb8f936df231ad63b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:50.792799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1283/2591] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:50 2022] PUT /identity/v3/projects/492e20665a634eb3b48c6f3dadc5b57f/users/f5d8077b412241cd908ba839258c5024/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:50.804796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:50.805091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:50.805461 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:50.838323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] MFA Rules not processed for user `f5d8077b412241cd908ba839258c5024`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:50.839640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:50.932022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1309/2592] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:50.940025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:50.942547 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:51.042447 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.043908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:51.053014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e9198-3a0b-47e3-a441-737408b22ab7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.164698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.165501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.165784 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.166193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.186553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.188238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653cab6-0020-434f-9d05-7e4416a5296e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.222329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb5f1b-1336-4938-a0a1-4a8252671a2e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.226950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.227749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.228687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.229037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] PATH_INFO: `/v3/projects/fdbbb020a1944de2a8fa16379f8cec7d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.230412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbbb020a1944de2a8fa16379f8cec7d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.231497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1284/2593] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:51.232810 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.305167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbbb020a1944de2a8fa16379f8cec7d', 'request_id': 'req-d547f0ec-eaff-439f-87e7-5cf776f0e588'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:51.314169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbbb020a1944de2a8fa16379f8cec7d', 'request_id': 'req-d547f0ec-eaff-439f-87e7-5cf776f0e588'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:51.320765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbbb020a1944de2a8fa16379f8cec7d', 'request_id': 'req-d547f0ec-eaff-439f-87e7-5cf776f0e588'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:51.356085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] The token cache is being invalidate because project fdbbb020a1944de2a8fa16379f8cec7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:51.356602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:51.357526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d547f0ec-eaff-439f-87e7-5cf776f0e588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:51.359610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1310/2594] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:51 2022] DELETE /identity/v3/projects/fdbbb020a1944de2a8fa16379f8cec7d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:13:51.364487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.364847 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.365188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.389759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:51.391427 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:51.396869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:51.511734 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1285/2595] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:51.518470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.518786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.519018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.550824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:51.552105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:51.565088 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1311/2596] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:51.566448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.578985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16584f8f-024a-4aa0-a523-aa04babcd07d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.586458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.600064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.600534 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.603262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.604006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:13:51.604324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.606789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.624667 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1286/2597] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:13:51.626947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.647085 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fc78-649f-44ca-9c2d-61ff6e0aa9aa None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.653044 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.653817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.654444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.654637 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.655431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.657851 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.673958 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:51.701969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1312/2598] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:51 2022] POST /identity/v3/projects => generated 401 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:51.706271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.726185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f959d0-b71e-44e3-b6ab-83c21c941f71 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.866106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.866739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.867302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.867580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.868447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.870747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.913793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1287/2599] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:51 2022] POST /identity/v3/users => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:51.919317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.950175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c487ccb-c73b-421a-bb21-fe988474c132 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:51.958957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:51.959698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:51.960146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:51.960495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:51.961229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:51.964095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:51.978392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1313/2600] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:51.982997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:51.995459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127ea5b-dea4-4273-a41b-db6b61bd5d03 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.001531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.001897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.002256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.002574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.003219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.004863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3a627-441f-4ecb-982f-3a695501380d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.017371 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1288/2601] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:52.024954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.037962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40172e-f5ca-4344-8d91-b23d88de9f10 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.044646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.045046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.045319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.045588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] PATH_INFO: `/v3/projects/3b0378de2caf44a6b88d74427c329130/users/9105c685d98b4315a0b024da2ea29b85/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.062446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0378de2caf44a6b88d74427c329130, user_id=9105c685d98b4315a0b024da2ea29b85, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.065222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.085158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1314/2602] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:52 2022] PUT /identity/v3/projects/3b0378de2caf44a6b88d74427c329130/users/9105c685d98b4315a0b024da2ea29b85/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:52.097031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.097610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.097951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.131452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] MFA Rules not processed for user `9105c685d98b4315a0b024da2ea29b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:52.132835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:52.246727 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.247288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1289/2603] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:52 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:52.368302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7ce87-ecbc-45a7-9b4d-449196a0f3f0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.377418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.379008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.379703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.380450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.381671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.385392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.424237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1315/2604] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:52 2022] POST /identity/v3/projects => generated 401 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:52.428146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.438219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:52.441599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f756777-2256-4df6-8f7d-2b77902c64bc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.515508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.516186 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.516536 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.516949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.517640 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.519693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.548808 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1290/2605] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:52 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:13:52.549718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.561379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046ee79-a82b-41b8-8d67-098d256a932d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.565915 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.566635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.566973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.567936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.568062 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.570373 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.583019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1316/2606] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:13:52.587780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.598103 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff789d-8a2f-4961-8554-ae9b23d311e4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.603088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.603751 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.604028 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.604297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] PATH_INFO: `/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd/users/b42d8ff9edd645e5bc07aeee47cfcf83/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.623191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4beddc8ca84d8080925b8ac7e500dd, user_id=b42d8ff9edd645e5bc07aeee47cfcf83, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.627262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.642786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1291/2607] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:52 2022] PUT /identity/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd/users/b42d8ff9edd645e5bc07aeee47cfcf83/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:52.647090 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.657992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be22f7-8dfc-482d-ac78-ced7fdd7058e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.726312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.727482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.728035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.728035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.728801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.731781 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d741e-fdd6-4600-b866-59b2194307fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.752353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.752865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1317/2608] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:52.764998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46cc98-784d-4e57-a51f-b6c1fd6e4a1a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:52.771353 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:52.772026 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.772308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.772582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] PATH_INFO: `/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd/users/b42d8ff9edd645e5bc07aeee47cfcf83/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.775257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4beddc8ca84d8080925b8ac7e500dd, user_id=b42d8ff9edd645e5bc07aeee47cfcf83, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:52.778145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:52.790855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1292/2609] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:52 2022] PUT /identity/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd/users/b42d8ff9edd645e5bc07aeee47cfcf83/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:52.803496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:52.803890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:52.804168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:52.835801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] MFA Rules not processed for user `b42d8ff9edd645e5bc07aeee47cfcf83`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:52.837316 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:52.854005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:52.921079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1318/2610] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:13:52.932954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:52.935004 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:53.000314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38cc80-5c10-4e17-8294-cc32f0339a2c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:53.015261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:53.015261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:53.015261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:53.015261 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:53.015670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:53.016652 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:53.028413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1293/2611] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:52 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:53.123108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:53.139400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febeff6-ef6a-4066-86a7-0060ffa1168e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:53.144383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:53.145005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:53.145281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:53.145577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:53.214133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:53.215808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66911766-03ef-47eb-bf80-7d5d18500903 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:53.225704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1319/2612] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:53 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:57.615966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:57.694630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75197-00f9-42e7-b2d1-31b6533aea1e service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:57.700017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:57.700729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:57.701204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:57.701610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] PATH_INFO: `/v3/users/9105c685d98b4315a0b024da2ea29b85` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:57.702850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9105c685d98b4315a0b024da2ea29b85)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:57.704935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:57.707247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:57.759848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9105c685d98b4315a0b024da2ea29b85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:57.760747 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9105c685d98b4315a0b024da2ea29b85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:57.768210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9105c685d98b4315a0b024da2ea29b85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:57.776493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9105c685d98b4315a0b024da2ea29b85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:57.777128 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9105c685d98b4315a0b024da2ea29b85'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:57.796843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1294/2613] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:57 2022] DELETE /identity/v3/users/9105c685d98b4315a0b024da2ea29b85 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:57.799994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:57.800580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:57.800580 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:57.820927 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:57.822457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:57.885584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1320/2614] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:57.895949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:57.897444 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:57.962341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24fada-e57c-4c5f-9cfd-9693584b545c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:57.966119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:57.966724 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:57.967005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:57.967309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:57.976474 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:57.978206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:57.988304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1295/2615] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:13:58.296216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:58.298436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:58.321254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ffd52-fb98-4f2c-b940-80ef3a24b829 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:58.326820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:58.327622 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:58.327992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:58.328401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] PATH_INFO: `/v3/projects/3b0378de2caf44a6b88d74427c329130` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:58.329696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0378de2caf44a6b88d74427c329130)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:58.331798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:58.388049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:58.392261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0378de2caf44a6b88d74427c329130', 'request_id': 'req-120e9fe2-a442-4f54-89fe-f549e289afa7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.401874 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0378de2caf44a6b88d74427c329130', 'request_id': 'req-120e9fe2-a442-4f54-89fe-f549e289afa7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.409627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0378de2caf44a6b88d74427c329130', 'request_id': 'req-120e9fe2-a442-4f54-89fe-f549e289afa7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.426216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] The token cache is being invalidate because project 3b0378de2caf44a6b88d74427c329130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:58.426608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.427412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.429189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1321/2616] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:58 2022] DELETE /identity/v3/projects/3b0378de2caf44a6b88d74427c329130 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:58.433632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:58.447054 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba2a7-e3f0-42ce-8e60-441a410716f0 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:58.520099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:58.520707 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:58.520985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:58.521260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] PATH_INFO: `/v3/users/b42d8ff9edd645e5bc07aeee47cfcf83` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:58.522420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42d8ff9edd645e5bc07aeee47cfcf83)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:58.524829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:58.564493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:58.591284 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42d8ff9edd645e5bc07aeee47cfcf83'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.591928 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42d8ff9edd645e5bc07aeee47cfcf83'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.598875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42d8ff9edd645e5bc07aeee47cfcf83'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.607508 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42d8ff9edd645e5bc07aeee47cfcf83'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.608121 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e19ac057-5a29-40e8-98eb-ba9f0132164e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42d8ff9edd645e5bc07aeee47cfcf83'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:58.626151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1296/2617] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:13:58 2022] DELETE /identity/v3/users/b42d8ff9edd645e5bc07aeee47cfcf83 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:13:58.929894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:58.932448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:58.950587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e9fe2-a442-4f54-89fe-f549e289afa7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.017471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.018160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.018440 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.018736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] PATH_INFO: `/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.019880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4beddc8ca84d8080925b8ac7e500dd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.021881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.069151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4beddc8ca84d8080925b8ac7e500dd', 'request_id': 'req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:59.075335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4beddc8ca84d8080925b8ac7e500dd', 'request_id': 'req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:59.080608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4beddc8ca84d8080925b8ac7e500dd', 'request_id': 'req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:59.095359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] The token cache is being invalidate because project 9e4beddc8ca84d8080925b8ac7e500dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:13:59.095569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:59.096194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9d9ede70-05d0-4f43-9b3d-82cb7dabce34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:13:59.097693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1322/2618] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:13:58 2022] DELETE /identity/v3/projects/9e4beddc8ca84d8080925b8ac7e500dd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:59.103204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.103493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.103768 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.125085 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:59.126296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:59.187749 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1297/2619] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:59.195670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.195954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.196277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.217695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:59.218899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:59.228766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1323/2620] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:59.233502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.244107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14361611-6e67-44cf-935b-8c8b0f60e336 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.249255 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.249848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.250119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.250389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.251057 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:13:59.251364 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.252984 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.265126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1298/2621] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:13:59.269710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.279709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4b828-3bd0-4316-b1d1-6a5414025906 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.285506 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.286117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.286398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.286672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.287458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.289433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.321408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1324/2622] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:59.325721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.335882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759e933-a80b-4fe5-a3df-b427e8093e1d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.395979 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.396524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.396792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.397066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.397698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.399924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.424793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1299/2623] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/users => generated 304 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:59.429223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.440310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f75116-7da7-46c6-95d0-29b2ac6bca0b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.446366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.446960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.447283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.447579 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.448215 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.449830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.459953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1325/2624] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:59.464397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.474414 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695622b-a340-4e2a-82f2-357864ba05f6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.479840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.480423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.481092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.481361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.482001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.483642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475f9b2-9923-478e-b2db-4a3af4088350 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.494382 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1300/2625] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:13:59.498860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.509400 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab0b-cf42-426c-83ba-955d8297d094 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.514528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.515345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.515345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.516326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] PATH_INFO: `/v3/projects/f7ddd6dee4434cd2a4cf0a8ed46760ad/users/a579f65913ec4ed398cbf543b843c50e/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.532521 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ddd6dee4434cd2a4cf0a8ed46760ad, user_id=a579f65913ec4ed398cbf543b843c50e, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.535198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.547277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:59.552129 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1326/2626] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:59 2022] PUT /identity/v3/projects/f7ddd6dee4434cd2a4cf0a8ed46760ad/users/a579f65913ec4ed398cbf543b843c50e/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:59.563927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3488-9313-4a50-a408-6896b3663375 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.564264 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3488-9313-4a50-a408-6896b3663375 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.564538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3488-9313-4a50-a408-6896b3663375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.578899 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:13:59.602865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-360e3488-9313-4a50-a408-6896b3663375 None None] MFA Rules not processed for user `a579f65913ec4ed398cbf543b843c50e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:13:59.604347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-360e3488-9313-4a50-a408-6896b3663375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:13:59.677317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1301/2627] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:13:59.681925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.745126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5eff7-025f-4b8d-8fe2-1fe1014ba1e7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.750166 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.750775 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.751050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.751359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.752011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.753924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.784966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1327/2628] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:13:59.790027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3488-9313-4a50-a408-6896b3663375 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.801566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3488-9313-4a50-a408-6896b3663375 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.868303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.868915 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.869197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.869468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.870118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.872108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.895287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1302/2629] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:13:59 2022] POST /identity/v3/users => generated 310 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:13:59.899802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.910294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514276e-332a-48ee-abbe-43a76304b108 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.915704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.916310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.916582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.917005 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.917650 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.919265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.930963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1328/2630] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:13:59.935596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:13:59.946213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9a8f-ea65-4367-a9d1-796a2c34f92e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:13:59.951524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:13:59.952283 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:13:59.952694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:13:59.953145 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] PATH_INFO: `/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156/users/788d093045e8437686f1ca44a57d3694/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:13:59.973313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2c5bb26125412d9d5ab83fcfdb4156, user_id=788d093045e8437686f1ca44a57d3694, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:13:59.976047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:13:59.991476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1303/2631] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:13:59 2022] PUT /identity/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156/users/788d093045e8437686f1ca44a57d3694/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:13:59.996412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:00.006907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989687e-4202-417e-a44b-92b458bafeb5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:00.079611 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:00.080248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:00.080530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:00.080801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:00.081459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:00.083104 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd157a-d6de-4c92-a3d9-c10dabd4d6a3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:00.107121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1329/2632] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:00.113743 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:00.124308 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e981acf-e44a-49da-9373-483cc3441976 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:00.128989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:00.129657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:00.129933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:00.130180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] PATH_INFO: `/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156/users/788d093045e8437686f1ca44a57d3694/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:00.132248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2c5bb26125412d9d5ab83fcfdb4156, user_id=788d093045e8437686f1ca44a57d3694, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:00.135002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:00.147660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1304/2633] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:00 2022] PUT /identity/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156/users/788d093045e8437686f1ca44a57d3694/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:00.157855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:00.158145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:00.158422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:00.187488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] MFA Rules not processed for user `788d093045e8437686f1ca44a57d3694`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:00.188685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:00.250007 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:00.260491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1330/2634] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:00.277593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:00.280089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:00.342573 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:00.347182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef6c6-04eb-4c6a-adfe-e71d051539ab admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:00.352602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:00.353091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:00.353379 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:00.353864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:00.364142 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:00.365693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:00.375679 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1305/2635] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:00.538893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:00.549302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0bca8-648d-4186-b893-42ee3dbcc630 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:00.553726 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:00.554448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:00.554788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:00.555197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:00.615529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:00.617023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d1cc-46d9-47de-bb17-5f0e8a1c6b88 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:00.626249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1331/2636] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:01.900346 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:01.901941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:01.977319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d72b2-8aa8-4266-a115-4e844fb5f51d service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:01.982629 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:01.983300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:01.983582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:01.983851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] PATH_INFO: `/v3/users/f5d8077b412241cd908ba839258c5024` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:01.985092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d8077b412241cd908ba839258c5024)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:01.987064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:02.037552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d8077b412241cd908ba839258c5024'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.038244 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d8077b412241cd908ba839258c5024'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.045256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d8077b412241cd908ba839258c5024'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.054398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d8077b412241cd908ba839258c5024'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.055589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d8077b412241cd908ba839258c5024'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.073590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1306/2637] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:01 2022] DELETE /identity/v3/users/f5d8077b412241cd908ba839258c5024 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:02.078181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:02.078647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:02.078757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:02.100252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:02.101524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:02.162616 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1332/2638] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:02.173081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:02.174492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:02.234760 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3920-38b1-44fe-bbc2-1966a8e2eef2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:02.240999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:02.241636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:02.242511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:02.242511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:02.253676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:02.255830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:02.266092 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1307/2639] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:14:02.569960 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:02.571838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:02.590762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9abf5-683d-488f-a459-196f9e42984b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:02.598510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:02.599246 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:02.599525 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:02.599793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] PATH_INFO: `/v3/projects/492e20665a634eb3b48c6f3dadc5b57f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:02.600950 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=492e20665a634eb3b48c6f3dadc5b57f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:02.603619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:02.658264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492e20665a634eb3b48c6f3dadc5b57f', 'request_id': 'req-5dff8399-0001-4aee-89f9-ba152cac5069'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.668399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492e20665a634eb3b48c6f3dadc5b57f', 'request_id': 'req-5dff8399-0001-4aee-89f9-ba152cac5069'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.676197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492e20665a634eb3b48c6f3dadc5b57f', 'request_id': 'req-5dff8399-0001-4aee-89f9-ba152cac5069'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.694034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] The token cache is being invalidate because project 492e20665a634eb3b48c6f3dadc5b57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:02.694330 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.694946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5dff8399-0001-4aee-89f9-ba152cac5069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:02.696520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1333/2640] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:02 2022] DELETE /identity/v3/projects/492e20665a634eb3b48c6f3dadc5b57f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:08.165511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:08.167602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:08.183252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d38f3-b9be-44f7-ab9c-3069ea2f6abb service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:08.195740 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:08.259957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:08.260730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:08.260975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:08.261262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] PATH_INFO: `/v3/users/a579f65913ec4ed398cbf543b843c50e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:08.262602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a579f65913ec4ed398cbf543b843c50e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:08.264635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:08.329882 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a579f65913ec4ed398cbf543b843c50e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:08.330710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a579f65913ec4ed398cbf543b843c50e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:08.339969 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a579f65913ec4ed398cbf543b843c50e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:08.353566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a579f65913ec4ed398cbf543b843c50e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:08.354326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a579f65913ec4ed398cbf543b843c50e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:08.376499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1308/2641] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:08 2022] DELETE /identity/v3/users/a579f65913ec4ed398cbf543b843c50e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:08.382071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:08.382825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:08.382825 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:08.407882 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:08.409152 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:08.426530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:08.489303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1334/2642] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:14:08.499948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:08.501511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:08.572662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e68c0-789b-4615-b632-8607e8ca66a2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:08.577631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:08.580373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:08.580373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:08.580373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:08.592048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:08.593817 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:08.608260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1309/2643] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:09.086220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:09.087703 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:09.101474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c474ef7-67a8-4184-bac4-1c1181e43b26 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:09.109260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:09.111183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:09.111465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:09.111749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] PATH_INFO: `/v3/projects/f7ddd6dee4434cd2a4cf0a8ed46760ad` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:09.113045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ddd6dee4434cd2a4cf0a8ed46760ad)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:09.114988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:09.181576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ddd6dee4434cd2a4cf0a8ed46760ad', 'request_id': 'req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.188842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ddd6dee4434cd2a4cf0a8ed46760ad', 'request_id': 'req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.194578 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ddd6dee4434cd2a4cf0a8ed46760ad', 'request_id': 'req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.209842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] The token cache is being invalidate because project f7ddd6dee4434cd2a4cf0a8ed46760ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:09.210151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.210794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.212798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1335/2644] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:09 2022] DELETE /identity/v3/projects/f7ddd6dee4434cd2a4cf0a8ed46760ad => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:09.216852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:09.226987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231005-8530-422a-be05-86ba0319b3d8 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:09.264925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:09.298479 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:09.299184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:09.299466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:09.299742 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] PATH_INFO: `/v3/users/788d093045e8437686f1ca44a57d3694` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:09.300936 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788d093045e8437686f1ca44a57d3694)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:09.302924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:09.352555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788d093045e8437686f1ca44a57d3694'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.353127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788d093045e8437686f1ca44a57d3694'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.360575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788d093045e8437686f1ca44a57d3694'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.369003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788d093045e8437686f1ca44a57d3694'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.369910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3266f9db-6c75-46c4-8999-8f93bf76d670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788d093045e8437686f1ca44a57d3694'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:09.388136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1310/2645] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:09 2022] DELETE /identity/v3/users/788d093045e8437686f1ca44a57d3694 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:10.070876 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.072430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:10.089497 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87cfc3-4dfe-49eb-8ecc-8a43980fb2d8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.158045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.158788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.159159 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.159495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] PATH_INFO: `/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.160730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2c5bb26125412d9d5ab83fcfdb4156)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.162732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.213304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:10.217820 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2c5bb26125412d9d5ab83fcfdb4156', 'request_id': 'req-fceb4c0d-1190-4e0c-8b36-20827644d9cf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:10.226885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2c5bb26125412d9d5ab83fcfdb4156', 'request_id': 'req-fceb4c0d-1190-4e0c-8b36-20827644d9cf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:10.235343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2c5bb26125412d9d5ab83fcfdb4156', 'request_id': 'req-fceb4c0d-1190-4e0c-8b36-20827644d9cf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:10.250788 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] The token cache is being invalidate because project 2e2c5bb26125412d9d5ab83fcfdb4156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:10.251452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:10.252151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-fceb4c0d-1190-4e0c-8b36-20827644d9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:10.253716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1336/2646] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:10 2022] DELETE /identity/v3/projects/2e2c5bb26125412d9d5ab83fcfdb4156 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:14:10.259482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.260012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.260100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.292490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:10.295006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:10.378384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1311/2647] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:14:10.379795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.380082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.380355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.400902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:10.402531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:10.413859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1337/2648] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:10.415865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.427607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb09a-c1e4-41df-b8c3-45b107e600cd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.433437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.434287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.434985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.435311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.436204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:14:10.436680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.438305 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.452439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1312/2649] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:14:10.457079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.467250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ace97-2e24-464d-ad54-03bf9b60cc80 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.473418 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.474122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.474403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.474674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.475359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.477519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.510919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1338/2650] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:10 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:10.514948 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.528668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c530b-d268-4fd0-9f8a-45158b0fe79b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.576607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:10.602779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.603437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.603712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.603978 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.604708 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.606937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.631544 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1313/2651] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:10 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:14:10.636222 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.649234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8174ca-42ea-40e4-add1-8070fadc4454 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.654533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.655165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.655445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.655714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.656494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.657956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.669760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1339/2652] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:14:10.673962 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.685232 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db9f1-ecf5-4245-87b2-932517b7621d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.690312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.690994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.691334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.691600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.692392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.694022 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaf344-903b-41d4-b1bd-318537f2c491 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.710593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:10.710950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1314/2653] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:10.721194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45a240-e22b-4ec7-907a-a5045c21b07c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:10.726026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:10.726849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.727238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.727585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] PATH_INFO: `/v3/projects/453755eae4d449b387bf4ed93fc2ea2e/users/b55c6e993f2e4aa1959cffa836be189a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.873105 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453755eae4d449b387bf4ed93fc2ea2e, user_id=b55c6e993f2e4aa1959cffa836be189a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:10.875786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:10.890627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1340/2654] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:10 2022] PUT /identity/v3/projects/453755eae4d449b387bf4ed93fc2ea2e/users/b55c6e993f2e4aa1959cffa836be189a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:10.902431 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:10.902718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:10.903094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:10.933401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] MFA Rules not processed for user `b55c6e993f2e4aa1959cffa836be189a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:10.934411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:11.005725 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1315/2655] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:11.010929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.078360 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c1d-2764-4278-976b-789fd8aba653 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.083581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.084180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.084433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.084731 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.085368 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.087303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.116918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1341/2656] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:11 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:11.120833 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.133334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318593-3cbf-4ec8-b3ee-cfec33deb629 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.201507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.202171 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.202449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.202719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.203416 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.205774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.227700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1316/2657] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:11 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:11.232432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.242827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991647b-ccdb-4052-8244-b037ac0edbd6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.248661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.249288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.249565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.249858 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.250502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.252165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.262526 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1342/2658] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:11.266914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.277483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9d73e-c7a1-4940-9d6e-f43198571c35 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.282488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.283206 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.283488 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.283759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] PATH_INFO: `/v3/projects/18fd843d8dfb4dc39f88e14b83859af0/users/4874ebb97d104eb8ae3cd67b8000786a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.300634 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fd843d8dfb4dc39f88e14b83859af0, user_id=4874ebb97d104eb8ae3cd67b8000786a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.303468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.318418 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1317/2659] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:11 2022] PUT /identity/v3/projects/18fd843d8dfb4dc39f88e14b83859af0/users/4874ebb97d104eb8ae3cd67b8000786a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:14:11.322636 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.333042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31a4e1-464b-4046-a2de-fce150fc086f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.382181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:11.402184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.402789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.403112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.403388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.404031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.405717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1936b-48a1-4798-b5df-aac1d7ea8618 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.416385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1343/2660] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:11.421310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.433894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b5730-c176-404e-bad9-f7c6c9edad87 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.440439 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.441180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.441516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.441792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] PATH_INFO: `/v3/projects/18fd843d8dfb4dc39f88e14b83859af0/users/4874ebb97d104eb8ae3cd67b8000786a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.444032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fd843d8dfb4dc39f88e14b83859af0, user_id=4874ebb97d104eb8ae3cd67b8000786a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.447644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.464480 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1318/2661] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:11 2022] PUT /identity/v3/projects/18fd843d8dfb4dc39f88e14b83859af0/users/4874ebb97d104eb8ae3cd67b8000786a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:11.475363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15222c53-173a-4062-8470-404e41c49190 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.475601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15222c53-173a-4062-8470-404e41c49190 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.475900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15222c53-173a-4062-8470-404e41c49190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.508120 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-15222c53-173a-4062-8470-404e41c49190 None None] MFA Rules not processed for user `4874ebb97d104eb8ae3cd67b8000786a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:11.509259 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15222c53-173a-4062-8470-404e41c49190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:11.583476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1344/2662] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:11.597411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.599700 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:11.607548 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:11.667464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa131f-7d29-4cef-a85a-0c7e351d3baa admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.673884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.674607 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.674812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.675246 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.688262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.689713 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.699894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1319/2663] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:11.798827 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15222c53-173a-4062-8470-404e41c49190 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:11.811433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15222c53-173a-4062-8470-404e41c49190 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:11.815442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:11.816046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:11.816326 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:11.816597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:11.874034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:11.875504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0965a87-602a-442d-a731-b1cc6a112a29 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:11.884607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1345/2664] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:16.752285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:16.819663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdbf07-dc2e-4beb-952b-f19c99590044 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:16.825623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:16.826317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:16.826507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:16.826796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] PATH_INFO: `/v3/users/b55c6e993f2e4aa1959cffa836be189a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:16.827963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55c6e993f2e4aa1959cffa836be189a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:16.829919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:16.885449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55c6e993f2e4aa1959cffa836be189a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:16.886153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55c6e993f2e4aa1959cffa836be189a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:16.893044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55c6e993f2e4aa1959cffa836be189a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:16.902139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55c6e993f2e4aa1959cffa836be189a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:16.902799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55c6e993f2e4aa1959cffa836be189a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:16.923311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1320/2665] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:16 2022] DELETE /identity/v3/users/b55c6e993f2e4aa1959cffa836be189a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:16.927910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:16.928416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:16.928529 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:16.948708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:16.949908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:17.023614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1346/2666] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:17.034638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:17.036516 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:17.069257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:17.102907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105196-5c97-4577-a1a8-90d7ba392b3b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:17.107198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:17.107840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:17.108126 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:17.108335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:17.119645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:17.121777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:17.139231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1321/2667] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:14:17.433713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:17.435348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:17.444145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:17.450826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3729831-f9e9-4bc7-b121-0f0202986379 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:17.456835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:17.457556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:17.457839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:17.458106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] PATH_INFO: `/v3/projects/453755eae4d449b387bf4ed93fc2ea2e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:17.459315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453755eae4d449b387bf4ed93fc2ea2e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:17.461292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:17.513861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453755eae4d449b387bf4ed93fc2ea2e', 'request_id': 'req-9839cfec-3a8c-4adc-907a-873c998d7fb6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.520586 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453755eae4d449b387bf4ed93fc2ea2e', 'request_id': 'req-9839cfec-3a8c-4adc-907a-873c998d7fb6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.527106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453755eae4d449b387bf4ed93fc2ea2e', 'request_id': 'req-9839cfec-3a8c-4adc-907a-873c998d7fb6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.542660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] The token cache is being invalidate because project 453755eae4d449b387bf4ed93fc2ea2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:17.542951 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.543625 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.545191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1347/2668] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:17 2022] DELETE /identity/v3/projects/453755eae4d449b387bf4ed93fc2ea2e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:17.549388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:17.560337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab1587-ebcd-4743-af81-873682f8edc4 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:17.631138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:17.631818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:17.632114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:17.632369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] PATH_INFO: `/v3/users/4874ebb97d104eb8ae3cd67b8000786a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:17.633619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4874ebb97d104eb8ae3cd67b8000786a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:17.635855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:17.687260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4874ebb97d104eb8ae3cd67b8000786a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.688072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4874ebb97d104eb8ae3cd67b8000786a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.695706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4874ebb97d104eb8ae3cd67b8000786a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.706528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4874ebb97d104eb8ae3cd67b8000786a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.707196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3dba97ed-be73-495d-a045-c597721e6e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4874ebb97d104eb8ae3cd67b8000786a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:17.726972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1322/2669] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:17 2022] DELETE /identity/v3/users/4874ebb97d104eb8ae3cd67b8000786a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:18.034873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.036410 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:18.051260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cfec-3a8c-4adc-907a-873c998d7fb6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.116508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.117148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.117421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.117690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] PATH_INFO: `/v3/projects/18fd843d8dfb4dc39f88e14b83859af0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.118826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=18fd843d8dfb4dc39f88e14b83859af0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.121272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.174197 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18fd843d8dfb4dc39f88e14b83859af0', 'request_id': 'req-cb9ef502-78ae-4cb3-a03c-79545452e56c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:18.181559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18fd843d8dfb4dc39f88e14b83859af0', 'request_id': 'req-cb9ef502-78ae-4cb3-a03c-79545452e56c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:18.196225 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18fd843d8dfb4dc39f88e14b83859af0', 'request_id': 'req-cb9ef502-78ae-4cb3-a03c-79545452e56c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:18.221311 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] The token cache is being invalidate because project 18fd843d8dfb4dc39f88e14b83859af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:18.221672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:18.222735 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cb9ef502-78ae-4cb3-a03c-79545452e56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:18.224945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1348/2670] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:18 2022] DELETE /identity/v3/projects/18fd843d8dfb4dc39f88e14b83859af0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:18.231350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.232152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.232152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.254323 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:18.255585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:18.301307 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:18.321934 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1323/2671] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:18.331122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.331412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.331711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.354783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:18.356353 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:18.366615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1349/2672] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:18.371476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.382061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc47f5-5f76-4731-8545-2f59b8257f13 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.387201 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.387834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.388116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.388387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.389038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:14:18.389342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.391223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.405809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1324/2673] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:14:18.410256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.421122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856c15d-7377-4951-b824-8b17dc36844e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.426719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.427287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.427567 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.427837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.428481 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.430393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.436115 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:18.467265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1350/2674] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:18.471461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.483176 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95db2-ac95-4fb8-950c-f82874205a3c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.548348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.548957 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.549239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.549510 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.550155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.552138 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.576620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1325/2675] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:14:18.580811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.591714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59101bb0-5078-4249-b005-44bbfc27a42c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.597082 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.597681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.597959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.598218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.598852 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.600483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.611069 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1351/2676] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:18.615258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.626491 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2173a5b-482e-4698-8f77-8e81fc1c4763 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.631777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.632375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.632673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.632945 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.633596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.635291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed30dc-3ec5-4e79-88c7-8d32cb2080ed admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.646153 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1326/2677] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:18.650603 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.661110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60804d6f-a44b-4c43-aa0f-f0c62cb23661 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.666059 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.667256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.667536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.667806 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] PATH_INFO: `/v3/projects/0dceaca24c8146a4bcf18968bef1e3d8/users/4d7d281ee8884ffbbf8e0c142d9e3c4a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.682892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dceaca24c8146a4bcf18968bef1e3d8, user_id=4d7d281ee8884ffbbf8e0c142d9e3c4a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.685577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.702642 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1352/2678] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:18 2022] PUT /identity/v3/projects/0dceaca24c8146a4bcf18968bef1e3d8/users/4d7d281ee8884ffbbf8e0c142d9e3c4a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:18.712879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.713154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.713467 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.743469 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] MFA Rules not processed for user `4d7d281ee8884ffbbf8e0c142d9e3c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:18.744691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:18.815278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1327/2679] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:18.820081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.896657 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107becf3-f972-42fb-9aa5-1134552ab092 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.901905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:18.902480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:18.902761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:18.903043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:18.903687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:18.905607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:18.937891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1353/2680] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:18.942288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:18.953633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da93573-a44e-4a9f-bc30-1f057ee35ab7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:18.995877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:19.015666 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.016271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.016569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.016853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.017552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.019528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.042838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1328/2681] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:18 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:19.046719 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.057504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c228-121c-448b-9bb9-275915c8b850 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.066083 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.066695 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.066975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.067286 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.067936 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.069711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.082240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1354/2682] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:19.087796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.100027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503226-f1c4-48f2-8a5d-6491090606db admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.107174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.108229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.108583 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.108905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] PATH_INFO: `/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b/users/405fe2c11c2d41b395d0ab0b6dcc8cbf/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.129025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9d5c64baf74fdcbfa003ed659e864b, user_id=405fe2c11c2d41b395d0ab0b6dcc8cbf, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.132286 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.148946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1329/2683] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:19 2022] PUT /identity/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b/users/405fe2c11c2d41b395d0ab0b6dcc8cbf/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:19.153576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.164932 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a43c4-3996-4098-8a47-3c6a8db695af admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.226131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.226653 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.226930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.227233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.227895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.229502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd27e2d-1294-4989-93e9-1956f91ce960 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.239585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1355/2684] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:19.244081 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.255343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77911b2-90b8-4c3c-9088-df6ff948b531 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.261066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.261714 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.261990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.262256 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] PATH_INFO: `/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b/users/405fe2c11c2d41b395d0ab0b6dcc8cbf/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.264241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9d5c64baf74fdcbfa003ed659e864b, user_id=405fe2c11c2d41b395d0ab0b6dcc8cbf, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.267314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.280250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1330/2685] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:19 2022] PUT /identity/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b/users/405fe2c11c2d41b395d0ab0b6dcc8cbf/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:19.290890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.291239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.292176 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.322378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] MFA Rules not processed for user `405fe2c11c2d41b395d0ab0b6dcc8cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:19.324269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:19.335093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:19.398383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1356/2686] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:19.411825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.413638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:19.477782 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d6ace-7dca-4a15-bbc2-975462479cda admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.482545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.483182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.483458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.483710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.493265 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.495676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.507397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1331/2687] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:19.587476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:19.598219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf170a9-d1d7-4b60-a6f4-bf87259e84c7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:19.602550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:19.603184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:19.603467 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:19.603828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:19.668142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:19.669583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a4047-cb08-46af-ad38-9dde852df1d1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:19.683376 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1357/2688] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:19.971099 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:20.036783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc5e51-6d5e-40cd-85ba-05e4b3956616 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:20.043109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:20.043732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:20.044008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:20.044274 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] PATH_INFO: `/v3/users/4d7d281ee8884ffbbf8e0c142d9e3c4a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:20.045412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7d281ee8884ffbbf8e0c142d9e3c4a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:20.047380 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:20.093144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:20.099797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7d281ee8884ffbbf8e0c142d9e3c4a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.100813 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7d281ee8884ffbbf8e0c142d9e3c4a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.109193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7d281ee8884ffbbf8e0c142d9e3c4a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.117741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7d281ee8884ffbbf8e0c142d9e3c4a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.118695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7d281ee8884ffbbf8e0c142d9e3c4a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.136942 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1332/2689] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:19 2022] DELETE /identity/v3/users/4d7d281ee8884ffbbf8e0c142d9e3c4a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:20.141723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:20.142255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:20.142255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:20.162592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:20.163840 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:20.231089 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1358/2690] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:20.240971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:20.242558 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:20.310664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7523ae-66f0-4073-81bc-7602eb555548 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:20.314917 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:20.315565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:20.315842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:20.316113 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:20.326443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:20.327939 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:20.337780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1333/2691] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:14:20.640796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:20.642267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:20.656382 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4d15a-0bb8-4658-9740-dc004d907062 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:20.661413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:20.662078 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:20.662355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:20.662621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] PATH_INFO: `/v3/projects/0dceaca24c8146a4bcf18968bef1e3d8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:20.663792 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dceaca24c8146a4bcf18968bef1e3d8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:20.665698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:20.710519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:20.717430 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dceaca24c8146a4bcf18968bef1e3d8', 'request_id': 'req-3629f8e8-61cd-40c2-8b54-7e662cbd372d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.724184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dceaca24c8146a4bcf18968bef1e3d8', 'request_id': 'req-3629f8e8-61cd-40c2-8b54-7e662cbd372d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.731086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dceaca24c8146a4bcf18968bef1e3d8', 'request_id': 'req-3629f8e8-61cd-40c2-8b54-7e662cbd372d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.746023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] The token cache is being invalidate because project 0dceaca24c8146a4bcf18968bef1e3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:20.746308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.746929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.748462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1359/2692] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:20 2022] DELETE /identity/v3/projects/0dceaca24c8146a4bcf18968bef1e3d8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:20.752594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:20.762620 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b5a9-1a25-4f8d-b50a-17527bc348cb service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:20.821736 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:20.822325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:20.822786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:20.823147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] PATH_INFO: `/v3/users/405fe2c11c2d41b395d0ab0b6dcc8cbf` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:20.824314 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] RBAC: Authorizing `identity:delete_user(user_id=405fe2c11c2d41b395d0ab0b6dcc8cbf)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:20.826296 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:20.875098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405fe2c11c2d41b395d0ab0b6dcc8cbf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.875746 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405fe2c11c2d41b395d0ab0b6dcc8cbf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.882472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405fe2c11c2d41b395d0ab0b6dcc8cbf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.886365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:20.893718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405fe2c11c2d41b395d0ab0b6dcc8cbf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.894658 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d32413ae-1bfd-41ab-a4f5-6d3d5f23716d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405fe2c11c2d41b395d0ab0b6dcc8cbf'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:20.914304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1334/2693] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:20 2022] DELETE /identity/v3/users/405fe2c11c2d41b395d0ab0b6dcc8cbf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:14:21.196328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.197895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:21.212649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629f8e8-61cd-40c2-8b54-7e662cbd372d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.273550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.274199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.274473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.274745 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] PATH_INFO: `/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.276011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9d5c64baf74fdcbfa003ed659e864b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.277969 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.328266 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9d5c64baf74fdcbfa003ed659e864b', 'request_id': 'req-ba663e05-d4c2-429d-81ab-545ef8f3a090'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:21.334232 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9d5c64baf74fdcbfa003ed659e864b', 'request_id': 'req-ba663e05-d4c2-429d-81ab-545ef8f3a090'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:21.339945 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9d5c64baf74fdcbfa003ed659e864b', 'request_id': 'req-ba663e05-d4c2-429d-81ab-545ef8f3a090'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:21.353492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] The token cache is being invalidate because project 5e9d5c64baf74fdcbfa003ed659e864b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:21.353890 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:21.354595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ba663e05-d4c2-429d-81ab-545ef8f3a090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:21.356423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1360/2694] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:21 2022] DELETE /identity/v3/projects/5e9d5c64baf74fdcbfa003ed659e864b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:21.361363 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.361855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.361992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.382507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:21.383745 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:21.447001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1335/2695] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:14:21.456047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.456329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.456593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.476324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:21.477527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:21.486207 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1361/2696] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:21.490460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.500702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd702e6-33a9-49ef-941a-6f23ae2d321f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.505606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.506226 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.506506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.506771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.507464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:14:21.507767 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.509377 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.521252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1336/2697] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:14:21.525721 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.535153 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d3a22-6ff4-40cd-9516-3f55e14a396b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.541136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.541718 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.541990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.542250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.542867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.544885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.578066 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1362/2698] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:21 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:21.582369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.595641 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342064-5185-4bac-bab4-3093784f74bc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.655084 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.655694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.655970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.656235 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.656869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.658783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.682354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1337/2699] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:21 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:21.688318 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.701920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bde35-7fca-459a-9687-cb3bb7c1715b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.706562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.707190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.707463 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.707724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.708354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.709963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.720396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1363/2700] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:21.724502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.735875 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d73bdf-ca93-4ad3-9c43-f1d2af4d0793 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.741409 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.742010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.742287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.742555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.743234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.744889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633978b-293d-403c-9fcd-56d89a8d2412 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.756260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1338/2701] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:21.761072 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.771277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5024ad-e13b-4c44-bb17-887e71c3db7a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:21.776323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:21.776962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.777238 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.777504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] PATH_INFO: `/v3/projects/96a2f3d229b84d12af986f9295d02abf/users/22d9fae4a7734dd8b75454a043fff93a/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.792849 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a2f3d229b84d12af986f9295d02abf, user_id=22d9fae4a7734dd8b75454a043fff93a, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:21.795517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:21.798588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:21.815663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1364/2702] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:21 2022] PUT /identity/v3/projects/96a2f3d229b84d12af986f9295d02abf/users/22d9fae4a7734dd8b75454a043fff93a/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:21.826907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:21.827216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:21.827489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:21.858930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] MFA Rules not processed for user `22d9fae4a7734dd8b75454a043fff93a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:21.860179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:21.896369 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:21.933738 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1339/2703] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:21.944892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:21.946740 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:22.006183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eaf91-b61d-46a9-913a-30ec0b3a1c02 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:22.010310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:22.010905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:22.011216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:22.011488 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:22.020679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:22.022125 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a49d22-8fb7-4c1f-838d-ea3fdc041c99 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:22.031184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1365/2704] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:31.930204 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=12 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:32.027253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=28 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:38.712069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:38.824675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144136a-5358-48c7-8dbd-8975d2ff95f7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:38.830529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:38.831381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:38.831692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:38.831951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] PATH_INFO: `/v3/users/22d9fae4a7734dd8b75454a043fff93a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:38.833067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d9fae4a7734dd8b75454a043fff93a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:38.835116 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:38.892527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d9fae4a7734dd8b75454a043fff93a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:38.893190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d9fae4a7734dd8b75454a043fff93a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:38.900522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d9fae4a7734dd8b75454a043fff93a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:38.909892 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d9fae4a7734dd8b75454a043fff93a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:38.910532 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d9fae4a7734dd8b75454a043fff93a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:38.928997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1340/2705] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:38 2022] DELETE /identity/v3/users/22d9fae4a7734dd8b75454a043fff93a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:38.933709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:38.934211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:38.934211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:38.985635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:38.986744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:39.056663 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1366/2706] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:39.066663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.068127 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:39.112015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:39.152151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3ff08-8aa4-48df-9164-7a790032b98f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.156438 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.157372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.157695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.157966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.168890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.170343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953650f9-aad7-4413-bc00-07d2214faa51 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.180378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1341/2707] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:39.472117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.473817 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:39.487842 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9584-db6c-49f2-ba35-0d342bb597a1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.493623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.494255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.494532 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.494801 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] PATH_INFO: `/v3/projects/96a2f3d229b84d12af986f9295d02abf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.496096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a2f3d229b84d12af986f9295d02abf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.497988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.547929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a2f3d229b84d12af986f9295d02abf', 'request_id': 'req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:39.553997 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a2f3d229b84d12af986f9295d02abf', 'request_id': 'req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:39.559432 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a2f3d229b84d12af986f9295d02abf', 'request_id': 'req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:39.573712 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] The token cache is being invalidate because project 96a2f3d229b84d12af986f9295d02abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:39.573946 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:39.574647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ff11cdbe-ff6e-4e25-8a73-4da85229a5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:39.576253 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1367/2708] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:39 2022] DELETE /identity/v3/projects/96a2f3d229b84d12af986f9295d02abf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:39.582674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991699-0101-47c2-a128-7cec0c366650 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.582965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991699-0101-47c2-a128-7cec0c366650 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.583287 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991699-0101-47c2-a128-7cec0c366650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.634703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-92991699-0101-47c2-a128-7cec0c366650 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:39.635970 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-92991699-0101-47c2-a128-7cec0c366650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:39.706931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1342/2709] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:39.716593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.716875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.717143 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.739803 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:39.741003 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:39.749749 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1368/2710] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:39.754449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92991699-0101-47c2-a128-7cec0c366650 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.764864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92991699-0101-47c2-a128-7cec0c366650 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.770778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.771448 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.771772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.772041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.772696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:14:39.773007 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.774644 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.791773 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1343/2711] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:14:39.798251 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.808590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7a21-6375-4261-8072-e168a8651079 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.814334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.814929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.815233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.815495 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.816122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.818012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.850722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1369/2712] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:39 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:39.855636 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.867180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5d683-92c5-44dc-8985-5abd2fc3bace admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.928457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.929059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.929344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.929606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.930239 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.932205 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.956976 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1344/2713] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:39 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:14:39.961953 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:39.973392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3543fa-a7ee-4ebe-9855-3f2ea11d59c6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:39.978204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:39.978794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:39.979073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:39.979357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:39.980313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:39.981902 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:39.991742 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1370/2714] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:39.995997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.006473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b69b-8c7b-42c7-9312-9360f20947a0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.011577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.012200 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.012477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.012718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.013388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.015017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56ade7-798d-43d0-8b5a-f7a35783ee98 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.025894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1345/2715] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:40.030022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.039959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810525e8-e157-4092-9c90-58d2444fc6ad admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.045342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.045978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.046255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.046524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] PATH_INFO: `/v3/projects/005a16ce617f465588e8342a1773a74d/users/e63f1059398547b3b3cc1b13f8dfa712/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.062058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005a16ce617f465588e8342a1773a74d, user_id=e63f1059398547b3b3cc1b13f8dfa712, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.064834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.067254 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:40.081561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1371/2716] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:40 2022] PUT /identity/v3/projects/005a16ce617f465588e8342a1773a74d/users/e63f1059398547b3b3cc1b13f8dfa712/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:14:40.148966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.149240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.149586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.169832 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:40.186765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] MFA Rules not processed for user `e63f1059398547b3b3cc1b13f8dfa712`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:40.188076 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:40.258625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1346/2717] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:40.263519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.340302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6340f-904d-416c-a91e-58f9f782cc17 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.346045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.346919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.347281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.347589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.348218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.350941 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.386021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1372/2718] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:40.390336 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.401590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3a6d2-df7a-49ff-97c3-643ae425e825 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.466897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.467564 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.467840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.468112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.468797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.470872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.496276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1347/2719] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:40.500671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.511590 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3eb37-0f0a-439d-b3c0-a707e1286ea7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.517221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.517808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.518081 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.518345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.518974 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.520607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.531807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1373/2720] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:40.536091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.546866 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30ba6-ebc2-4f6c-b587-2478de4b101e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.552023 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.552673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.552951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.553217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.553890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.555552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41129f-f5e9-44fc-8b4a-b864891f81a3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.566324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1348/2721] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:40.570796 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.581629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76193-bdb2-4c4e-a57a-8edf487eee4a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.587230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.587899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.588173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.588429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] PATH_INFO: `/v3/projects/f38feb7c22374bd7bd86cadfdf969d33/users/0d749ab9794048ee9c85385cbad7b161/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.605169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38feb7c22374bd7bd86cadfdf969d33, user_id=0d749ab9794048ee9c85385cbad7b161, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.607774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.623288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1374/2722] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:40 2022] PUT /identity/v3/projects/f38feb7c22374bd7bd86cadfdf969d33/users/0d749ab9794048ee9c85385cbad7b161/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:40.634061 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.634361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.634635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.669864 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] MFA Rules not processed for user `0d749ab9794048ee9c85385cbad7b161`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:40.671119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:40.739434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1349/2723] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:40.744202 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.809925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1b8f5-ad3f-4979-965b-2f8a8080546b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.816538 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.817148 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.817425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.817690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.818315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.820272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.850023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1375/2724] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:40.855999 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.863816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:40.869177 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed9c3c-e020-455a-ba22-6017dd9fdaac None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.929021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.929664 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.929950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.930229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.930891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.932863 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.956476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1350/2725] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:40 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:40.960560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:40.971865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df555c9-982d-4aad-89e1-2da9bd786b98 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:40.977047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:40.977628 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:40.977910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:40.978177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:40.978822 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:40.980785 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:40.991533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1376/2726] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:40.995692 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:41.006650 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55e05b-9987-4e88-8eed-eb5419e63cbb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:41.012417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:41.013108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:41.013402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:41.013702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] PATH_INFO: `/v3/projects/170e933de23d479d9786f13f112636da/users/c71d1427acdd4e49b7e5fa2396e5ade6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:41.028954 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170e933de23d479d9786f13f112636da, user_id=c71d1427acdd4e49b7e5fa2396e5ade6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:41.032082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:41.047972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1351/2727] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:40 2022] PUT /identity/v3/projects/170e933de23d479d9786f13f112636da/users/c71d1427acdd4e49b7e5fa2396e5ade6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:41.053322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:41.064006 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be788d7-0a70-4d9f-bbe1-0de2109376a0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:41.081823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:41.128018 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:41.128539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:41.128823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:41.129094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:41.129732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:41.131355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b594a0-e154-4adf-a0b5-d2efd37f7803 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:41.141332 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1377/2728] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:14:41.146511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:41.162338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755fbd1-9322-44dd-95eb-50cac0275183 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:41.170087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:41.170797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:41.171091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:41.171386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] PATH_INFO: `/v3/projects/170e933de23d479d9786f13f112636da/users/c71d1427acdd4e49b7e5fa2396e5ade6/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:41.173697 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170e933de23d479d9786f13f112636da, user_id=c71d1427acdd4e49b7e5fa2396e5ade6, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:41.176421 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:41.189196 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1352/2729] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:41 2022] PUT /identity/v3/projects/170e933de23d479d9786f13f112636da/users/c71d1427acdd4e49b7e5fa2396e5ade6/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:14:41.200807 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:41.201095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:41.201369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:41.230297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] MFA Rules not processed for user `c71d1427acdd4e49b7e5fa2396e5ade6`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:41.231559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:41.304933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1378/2730] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:41.317685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:41.319437 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:41.378249 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04074f49-ecd5-49a1-b9e7-f29ee0ae5852 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:41.382122 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:41.382696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:41.382977 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:41.383280 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:41.444741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:41.446197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:41.455415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1353/2731] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:43.007348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:43.008847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:43.021566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd4772-3cbb-4108-ab4a-68e4583b1595 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:43.025361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:43.025916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:43.026187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:43.026458 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:43.090601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:43.092080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:43.101570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1379/2732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:43.681965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:43.691392 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a0d1f-7e5e-4553-a197-a822adf87d2a service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:43.695785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:43.696384 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:43.696662 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:43.696931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:43.706528 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:43.708188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5beaee-b33e-4aa9-a307-81c89fa1f628 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:43.712012 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:43.719912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1354/2733] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:47.678516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:47.722963 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:47.749347 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a27f5b-7774-459c-9b67-dcbc88a45cd6 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:47.754412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:47.755043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:47.755333 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:47.755598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] PATH_INFO: `/v3/users/e63f1059398547b3b3cc1b13f8dfa712` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:47.756794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63f1059398547b3b3cc1b13f8dfa712)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:47.758717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:47.816404 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63f1059398547b3b3cc1b13f8dfa712'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:47.829850 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63f1059398547b3b3cc1b13f8dfa712'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:47.836809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63f1059398547b3b3cc1b13f8dfa712'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:47.837541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63f1059398547b3b3cc1b13f8dfa712'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:47.838184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63f1059398547b3b3cc1b13f8dfa712'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:47.854186 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1380/2734] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:47 2022] DELETE /identity/v3/users/e63f1059398547b3b3cc1b13f8dfa712 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:47.859718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:47.860224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:47.860338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:47.880406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:47.881623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:47.944797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1355/2735] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:47.956067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:47.957771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:48.038760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b263-c898-4ad1-8655-d4457b4bb96c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:48.042924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:48.043559 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:48.043838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:48.044108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:48.053914 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:48.055398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:48.066020 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1381/2736] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:48.341590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:48.343391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:48.357645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531ee3f-044c-4bef-b9c6-0a6505340e7c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:48.364805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:48.365449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:48.365862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:48.366824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] PATH_INFO: `/v3/projects/005a16ce617f465588e8342a1773a74d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:48.368459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] RBAC: Authorizing `identity:delete_project(project_id=005a16ce617f465588e8342a1773a74d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:48.370389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:48.421120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005a16ce617f465588e8342a1773a74d', 'request_id': 'req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.427173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005a16ce617f465588e8342a1773a74d', 'request_id': 'req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.433502 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005a16ce617f465588e8342a1773a74d', 'request_id': 'req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.450118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] The token cache is being invalidate because project 005a16ce617f465588e8342a1773a74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:48.450417 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.451172 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.452799 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1356/2737] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:48 2022] DELETE /identity/v3/projects/005a16ce617f465588e8342a1773a74d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:48.456990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:48.466900 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692eb644-10ea-4dbe-914c-3526422af456 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:48.510868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:48.534455 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:48.535147 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:48.535426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:48.535694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] PATH_INFO: `/v3/users/0d749ab9794048ee9c85385cbad7b161` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:48.536824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d749ab9794048ee9c85385cbad7b161)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:48.538739 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:48.587123 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d749ab9794048ee9c85385cbad7b161'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.599843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d749ab9794048ee9c85385cbad7b161'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.607269 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d749ab9794048ee9c85385cbad7b161'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.607975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d749ab9794048ee9c85385cbad7b161'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.608571 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d749ab9794048ee9c85385cbad7b161'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:48.622734 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1382/2738] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:48 2022] DELETE /identity/v3/users/0d749ab9794048ee9c85385cbad7b161 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:48.910366 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:48.911966 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:48.925674 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009fa0e-dd9b-4c5c-ba8a-83e76a08531b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:48.990492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:48.991378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:48.991691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:48.991963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] PATH_INFO: `/v3/projects/f38feb7c22374bd7bd86cadfdf969d33` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:48.993132 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38feb7c22374bd7bd86cadfdf969d33)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:48.995091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:49.019701 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:49.045554 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38feb7c22374bd7bd86cadfdf969d33', 'request_id': 'req-2f293e45-7c3c-43ef-8f3b-752c58655d44'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.052719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38feb7c22374bd7bd86cadfdf969d33', 'request_id': 'req-2f293e45-7c3c-43ef-8f3b-752c58655d44'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.059340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38feb7c22374bd7bd86cadfdf969d33', 'request_id': 'req-2f293e45-7c3c-43ef-8f3b-752c58655d44'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.073963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] The token cache is being invalidate because project f38feb7c22374bd7bd86cadfdf969d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:49.074251 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.075410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.076972 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1357/2739] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:48 2022] DELETE /identity/v3/projects/f38feb7c22374bd7bd86cadfdf969d33 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:49.081489 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:49.083016 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:49.096899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d149ffb-894b-441c-8717-4bc70c4b148a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:49.160492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:49.161163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.161442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.161716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] PATH_INFO: `/v3/users/c71d1427acdd4e49b7e5fa2396e5ade6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.162924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71d1427acdd4e49b7e5fa2396e5ade6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:49.164973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:49.215698 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71d1427acdd4e49b7e5fa2396e5ade6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.225867 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:49.228547 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71d1427acdd4e49b7e5fa2396e5ade6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.236863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71d1427acdd4e49b7e5fa2396e5ade6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.237600 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71d1427acdd4e49b7e5fa2396e5ade6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.238772 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e8c55d7e-3b40-4fc7-ac11-992e4c94445d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71d1427acdd4e49b7e5fa2396e5ade6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.254492 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1383/2740] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:49 2022] DELETE /identity/v3/users/c71d1427acdd4e49b7e5fa2396e5ade6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:49.536214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:49.537779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:49.551626 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293e45-7c3c-43ef-8f3b-752c58655d44 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:49.613932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:49.614572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.614850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.615154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] PATH_INFO: `/v3/projects/170e933de23d479d9786f13f112636da` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.616385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170e933de23d479d9786f13f112636da)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:49.618343 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:49.675580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170e933de23d479d9786f13f112636da', 'request_id': 'req-a476abf5-3a3e-4852-a6f5-5d9c9acda205'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.682575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170e933de23d479d9786f13f112636da', 'request_id': 'req-a476abf5-3a3e-4852-a6f5-5d9c9acda205'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.688880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170e933de23d479d9786f13f112636da', 'request_id': 'req-a476abf5-3a3e-4852-a6f5-5d9c9acda205'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.702752 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] The token cache is being invalidate because project 170e933de23d479d9786f13f112636da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:49.703082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.703690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a476abf5-3a3e-4852-a6f5-5d9c9acda205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:49.705373 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1358/2741] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:49 2022] DELETE /identity/v3/projects/170e933de23d479d9786f13f112636da => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:49.710114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.710398 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.710670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.731782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:49.732979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:49.793396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1384/2742] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:14:49.801730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.802014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.802281 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.823925 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:49.825253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:49.833796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1359/2743] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:49.838513 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:49.850894 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb063575-c98a-45c2-9afa-d0de95dc7538 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:49.873542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:49.874265 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.877412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.877699 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.878374 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:14:49.878705 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:49.880694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:49.894061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1385/2744] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:14:49.898838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:49.908702 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca31053-5f0b-4eb2-b38f-66cbe42e7c92 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:49.913492 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:49.914143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:49.914500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:49.914830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:49.917318 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:49.919442 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:49.952350 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1360/2745] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:14:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:14:49.956716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:49.967751 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0a75-9288-41ed-877d-b07b2eb5aa7b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:50.028378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:50.029017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.029299 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.029569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.030200 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:50.032491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:50.057344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1386/2746] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:14:49 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:14:50.059893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:50.070575 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4314917-6cee-4957-a560-44d4a4caf4c3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:50.076066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:50.076997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.077269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.077531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.078179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:50.080047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:50.090400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1361/2747] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:50.094366 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:50.105108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d6abd-4c43-4f55-9790-3e68f5e531c4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:50.110877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:50.111504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.111782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.112050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.112687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:50.114309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e80303-40c5-4eea-8755-ca6c62239e76 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:50.125436 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1387/2748] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:14:50.130109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:50.135789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:50.142611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ea33-9b3a-4b1b-a1d3-16b9f41458a4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:50.148685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:50.149344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.149623 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.149897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] PATH_INFO: `/v3/projects/6cf118eb09594dc998725295aa2f99eb/users/e7aac56947854d14a11e076727be90d7/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.164739 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf118eb09594dc998725295aa2f99eb, user_id=e7aac56947854d14a11e076727be90d7, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:50.167462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:50.183154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1362/2749] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:14:50 2022] PUT /identity/v3/projects/6cf118eb09594dc998725295aa2f99eb/users/e7aac56947854d14a11e076727be90d7/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:50.193892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.194193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.194457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.224067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] MFA Rules not processed for user `e7aac56947854d14a11e076727be90d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:50.225331 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:50.264619 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:50.303413 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1388/2750] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:14:50.316785 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:50.318777 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:50.409585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799e87-8394-4311-ad5e-d61e70644d6e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:50.413420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:50.414020 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:50.414297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:50.414565 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:50.423603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:50.425197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c707c-6533-4391-9558-dbc3e5a1e58e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:50.435315 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1363/2751] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:52.147962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:52.213175 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a00a1-4d99-405f-b0e1-11c16a5e9814 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:52.218195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:52.218848 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:52.219328 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:52.219457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] PATH_INFO: `/v3/users/e7aac56947854d14a11e076727be90d7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:52.220623 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7aac56947854d14a11e076727be90d7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:52.222536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:52.277024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7aac56947854d14a11e076727be90d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.290079 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7aac56947854d14a11e076727be90d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.296931 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7aac56947854d14a11e076727be90d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.297868 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7aac56947854d14a11e076727be90d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.298871 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7aac56947854d14a11e076727be90d7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.313267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1389/2752] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:14:52 2022] DELETE /identity/v3/users/e7aac56947854d14a11e076727be90d7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:14:52.318270 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:52.318791 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:52.318884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:52.338262 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:14:52.339499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:52.401919 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1364/2753] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:14:52.411924 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:52.413359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:52.469067 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1845ce-7ab0-4fe1-aa1b-e4e32725afd7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:52.474047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:52.474885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:52.475438 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:52.475821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:52.488664 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:52.492394 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:52.495272 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:52.511943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1390/2754] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:14:52.817494 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:14:52.818995 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:14:52.831540 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a7776-4882-4f19-b77d-a8a4cbdf418c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:14:52.836257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:14:52.836918 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:14:52.837197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:14:52.837466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] PATH_INFO: `/v3/projects/6cf118eb09594dc998725295aa2f99eb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:14:52.838631 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf118eb09594dc998725295aa2f99eb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:14:52.840587 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:14:52.892619 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf118eb09594dc998725295aa2f99eb', 'request_id': 'req-cc91862b-c20d-4326-b000-24e3057f176b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.899320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf118eb09594dc998725295aa2f99eb', 'request_id': 'req-cc91862b-c20d-4326-b000-24e3057f176b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.901356 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:14:52.906345 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf118eb09594dc998725295aa2f99eb', 'request_id': 'req-cc91862b-c20d-4326-b000-24e3057f176b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.920843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] The token cache is being invalidate because project 6cf118eb09594dc998725295aa2f99eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:14:52.921135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.921758 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cc91862b-c20d-4326-b000-24e3057f176b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:14:52.923387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1365/2755] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:14:52 2022] DELETE /identity/v3/projects/6cf118eb09594dc998725295aa2f99eb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:15:02.513928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:15:02.917893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=4,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:15:32.408111 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:15:32.409634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:15:32.505010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b1838-e6da-46bc-98a4-176b34283baf service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:15:32.573034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:15:32.573515 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:15:32.573786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:15:32.574049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:15:32.575970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:15:32.588042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:15:32.589641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:15:32.600160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1391/2756] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:15:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:15:42.595670 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=67 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:15:55.876340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:15:55.876986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:15:55.876986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:15:55.897335 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] MFA Rules not processed for user `62f5d4659eea4d91a689d709460396d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:15:55.898551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:15:55.961032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1366/2757] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:15:58.968160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:15:58.969906 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:15:59.090527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d31576-3e4d-4b10-8f2a-f9b2f9294349 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:15:59.097126 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:15:59.097736 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:15:59.098011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:15:59.098278 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:15:59.100203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:15:59.112184 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:15:59.113768 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:15:59.123534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1392/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:15:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:16:05.953537 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=25 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:16:09.119668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=46 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:16:28.007107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:16:28.008787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:16:28.069696 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5cbed-763e-48d7-8991-b23c5d1b99a1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:16:28.073645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:16:28.074272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:16:28.074549 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:16:28.074818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:16:28.076730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:16:28.090958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:16:28.092422 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ab009-5d07-4438-8c37-175db02b1325 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:16:28.103922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1367/2759] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:16:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:16:29.303537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:16:29.319101 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2d89-9404-47ea-889f-1c9dd6b305fe service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:16:29.325797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:16:29.326613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:16:29.326934 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:16:29.327297 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:16:29.330358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:16:29.346714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:16:29.348199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:16:29.360317 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1393/2760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:16:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:16:34.020013 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:16:34.020572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:16:34.020572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:16:34.041965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] MFA Rules not processed for user `e29d3d1942614652b3229888d04dc5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:16:34.043202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:16:34.103726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1368/2761] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:16:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:16:39.355107 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:16:44.094496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=51 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:39.207517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:39.218298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab3e31-ebb8-4e35-885e-775afcaec9e4 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:39.225095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:39.225697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:39.225992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:39.226248 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:39.228302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:39.239133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:39.240598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:39.249399 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1394/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:17:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:17:46.309878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:46.326078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57195744-a057-4a70-88c6-c1e13eee0acb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:46.333047 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:46.333718 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:46.334155 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:46.334435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:46.336582 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:46.350920 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:46.353240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:46.371149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1369/2763] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:17:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:17:49.245303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:55.071802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.071802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:55.146519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1b69b-e77d-46c8-ba4e-e6d2eaac0ba3 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.151838 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.152484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.152762 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.153036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] PATH_INFO: `/v3/users/e29d3d1942614652b3229888d04dc5bf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.154288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29d3d1942614652b3229888d04dc5bf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.156274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.216230 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29d3d1942614652b3229888d04dc5bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.228761 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29d3d1942614652b3229888d04dc5bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.235620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29d3d1942614652b3229888d04dc5bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.236345 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29d3d1942614652b3229888d04dc5bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.236998 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3b2a9d5e-1e45-41ef-bbe7-23da4e3cc599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29d3d1942614652b3229888d04dc5bf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.254379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1395/2764] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:17:55 2022] DELETE /identity/v3/users/e29d3d1942614652b3229888d04dc5bf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:17:55.446974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.460603 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e01bb-1008-493d-b8fc-56a1768e0d10 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.525501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.526136 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.526415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.526687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] PATH_INFO: `/v3/projects/b2f251186d5346d9ae213f018e299cb4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.527870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f251186d5346d9ae213f018e299cb4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.529796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.579579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f251186d5346d9ae213f018e299cb4', 'request_id': 'req-16958e27-64d1-4c95-90e8-5101bcadf74d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.585950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f251186d5346d9ae213f018e299cb4', 'request_id': 'req-16958e27-64d1-4c95-90e8-5101bcadf74d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.592005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f251186d5346d9ae213f018e299cb4', 'request_id': 'req-16958e27-64d1-4c95-90e8-5101bcadf74d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.607997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] The token cache is being invalidate because project b2f251186d5346d9ae213f018e299cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:17:55.608202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.608795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-16958e27-64d1-4c95-90e8-5101bcadf74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:17:55.610633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1370/2765] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:17:55 2022] DELETE /identity/v3/projects/b2f251186d5346d9ae213f018e299cb4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:17:55.615717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.616052 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.616267 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.637194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:17:55.638416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:55.700546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1396/2766] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:17:55.708994 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.709360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.709695 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.731823 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:17:55.733068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:55.743653 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1371/2767] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:17:55.748816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.764280 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2058bb-7cd5-4b98-8ff3-277a630b39ac None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.769614 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.770283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.770793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.771098 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.771773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:17:55.772075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.773686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.785550 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1397/2768] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:17:55.790151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.799542 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b99f6a-b548-45bf-8c33-996c1a57d669 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.804754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.805341 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.805615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.805907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.806529 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.808453 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.846197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1372/2769] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:17:55 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:17:55.850409 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.861050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d689c-f006-4645-8df9-96779e1871ae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.864818 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:55.919572 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.920163 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.920445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.920713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.921327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.923255 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.948459 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1398/2770] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:17:55 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:17:55.952774 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.963075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0a132-67cc-4ecd-8ef4-8189e09db614 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:55.967938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:55.968525 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:55.968797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:55.969062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:55.969729 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:55.971388 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:55.981862 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1373/2771] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:17:55.987491 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:55.999167 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fca32e-69bc-43c7-8829-8d9e89ec7153 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:56.005189 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:56.005794 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:56.006070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:56.006340 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:56.007058 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:56.008700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d8f23-2f6a-4037-9dfd-184bdf997b71 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:56.020717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1399/2772] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:17:56.025202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:56.035687 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96db3-b493-49da-8bcd-96e8303a0f16 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:56.041475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:56.042216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:56.042500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:56.042775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] PATH_INFO: `/v3/projects/b1ceffdf61d44ff9b7db06ae57395586/users/911896731640436aa9e8ea88ce323a6d/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:56.058770 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ceffdf61d44ff9b7db06ae57395586, user_id=911896731640436aa9e8ea88ce323a6d, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:56.061526 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:56.077390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1374/2773] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:17:56 2022] PUT /identity/v3/projects/b1ceffdf61d44ff9b7db06ae57395586/users/911896731640436aa9e8ea88ce323a6d/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:17:56.082581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec73f6-a4e4-443d-9ba4-fa69c5602d18 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:56.083190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec73f6-a4e4-443d-9ba4-fa69c5602d18 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:56.083561 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec73f6-a4e4-443d-9ba4-fa69c5602d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:56.102558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-54ec73f6-a4e4-443d-9ba4-fa69c5602d18 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:17:56.104084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-54ec73f6-a4e4-443d-9ba4-fa69c5602d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:56.174327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1400/2774] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:17:56.190096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:56.192164 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:56.199059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:56.255709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e1a-dff1-4daf-a275-63d6929f6893 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:56.260269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:56.260878 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:56.261157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:56.261428 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:56.271656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:56.273108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:56.282879 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1375/2775] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:17:56.968182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:56.968487 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:56.968694 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:56.998921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] MFA Rules not processed for user `911896731640436aa9e8ea88ce323a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:17:57.000183 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:57.077321 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1401/2776] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:17:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:17:57.078545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.089415 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71221e12-aab8-4fbf-8fa1-2dd28697e013 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.094411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.095135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.095472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.095816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.096458 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.098973 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.130810 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1376/2777] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:17:57 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:17:57.134638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.150565 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a2102-2341-445b-97eb-c098834ff800 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.163164 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:57.207843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.208766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.209048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.209312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.209978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.211916 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.241748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1402/2778] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:17:57 2022] POST /identity/v3/users => generated 307 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:17:57.243190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.254546 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292709f8-180b-48df-ad01-d7f761c84ff3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.260064 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.260709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.260987 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.261253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.261905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.263574 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.276086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1377/2779] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:17:57.283419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.295383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1a15-9a82-44bb-b5d9-39482356d7d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.299795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.300804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.301252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.301661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] PATH_INFO: `/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e/users/63ac06a6f4334208b634f6a728dcb30a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.318738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2043c59e8fc14ab39bf8c0bd02f1d10e, user_id=63ac06a6f4334208b634f6a728dcb30a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.324804 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.344672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1403/2780] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:17:57 2022] PUT /identity/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e/users/63ac06a6f4334208b634f6a728dcb30a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:17:57.349298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.361612 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793de0-55c8-4eb2-933a-652c2ba2ff14 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.457566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.459071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.459071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.459340 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.460117 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.461883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31527803-d0c4-4ceb-86af-a4b614f526d9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.478775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1378/2781] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:17:57.479797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:57.493730 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110b551-7917-498d-9653-e2671e1d9e59 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:57.499797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:57.499797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:57.499797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:57.499797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] PATH_INFO: `/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e/users/63ac06a6f4334208b634f6a728dcb30a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:57.519283 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2043c59e8fc14ab39bf8c0bd02f1d10e, user_id=63ac06a6f4334208b634f6a728dcb30a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:57.522319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:57.547275 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1404/2782] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:17:57 2022] PUT /identity/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e/users/63ac06a6f4334208b634f6a728dcb30a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:17:58.259197 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:58.260086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:58.260361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:58.298078 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] MFA Rules not processed for user `63ac06a6f4334208b634f6a728dcb30a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:17:58.299495 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:17:58.381690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:17:58.387865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1379/2783] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:17:58 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:17:59.856648 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:17:59.927338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae021ae-1c44-4d1f-97e2-b4f1d4013570 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:17:59.932060 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:17:59.932701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:17:59.932973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:17:59.933237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] PATH_INFO: `/v3/users/911896731640436aa9e8ea88ce323a6d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:17:59.934428 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=911896731640436aa9e8ea88ce323a6d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:17:59.936511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:17:59.994797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911896731640436aa9e8ea88ce323a6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.015169 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911896731640436aa9e8ea88ce323a6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.020557 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:00.024444 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911896731640436aa9e8ea88ce323a6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.025819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911896731640436aa9e8ea88ce323a6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.026531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911896731640436aa9e8ea88ce323a6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.043809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1405/2784] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:17:59 2022] DELETE /identity/v3/users/911896731640436aa9e8ea88ce323a6d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:00.231322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:00.243632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12554060-4a34-4c13-ad87-5f88301e50ae None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:00.323893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:00.324756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:00.325037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:00.325309 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] PATH_INFO: `/v3/projects/b1ceffdf61d44ff9b7db06ae57395586` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:00.326531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ceffdf61d44ff9b7db06ae57395586)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:00.328586 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:00.379940 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ceffdf61d44ff9b7db06ae57395586', 'request_id': 'req-e8daf987-8274-4f85-84e6-a6800a645836'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.387115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ceffdf61d44ff9b7db06ae57395586', 'request_id': 'req-e8daf987-8274-4f85-84e6-a6800a645836'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.392937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ceffdf61d44ff9b7db06ae57395586', 'request_id': 'req-e8daf987-8274-4f85-84e6-a6800a645836'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.407949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] The token cache is being invalidate because project b1ceffdf61d44ff9b7db06ae57395586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:00.408269 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.408914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.410523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1380/2785] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:00 2022] DELETE /identity/v3/projects/b1ceffdf61d44ff9b7db06ae57395586 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:00.414957 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:00.416417 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:00.430127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66cea-5c12-43cf-8f5f-229ab13632fc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:00.489576 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:00.490017 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:00.490445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:00.490892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] PATH_INFO: `/v3/users/63ac06a6f4334208b634f6a728dcb30a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:00.492203 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ac06a6f4334208b634f6a728dcb30a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:00.494421 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:00.545118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ac06a6f4334208b634f6a728dcb30a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.557716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ac06a6f4334208b634f6a728dcb30a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.564717 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ac06a6f4334208b634f6a728dcb30a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.565419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ac06a6f4334208b634f6a728dcb30a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.566043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9de080fd-3823-4bd1-9bd5-b8e771faf78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ac06a6f4334208b634f6a728dcb30a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.583329 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1406/2786] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:00 2022] DELETE /identity/v3/users/63ac06a6f4334208b634f6a728dcb30a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:00.761616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:00.763218 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:00.780140 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daf987-8274-4f85-84e6-a6800a645836 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:00.839316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:00.839937 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:00.840214 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:00.840481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] PATH_INFO: `/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:00.841625 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2043c59e8fc14ab39bf8c0bd02f1d10e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:00.843569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:00.877635 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:00.894477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2043c59e8fc14ab39bf8c0bd02f1d10e', 'request_id': 'req-7df55530-6462-48e8-98e8-5e5e644cdf7f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.900891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2043c59e8fc14ab39bf8c0bd02f1d10e', 'request_id': 'req-7df55530-6462-48e8-98e8-5e5e644cdf7f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.908032 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2043c59e8fc14ab39bf8c0bd02f1d10e', 'request_id': 'req-7df55530-6462-48e8-98e8-5e5e644cdf7f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.925424 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] The token cache is being invalidate because project 2043c59e8fc14ab39bf8c0bd02f1d10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:00.925715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.926470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7df55530-6462-48e8-98e8-5e5e644cdf7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:00.928294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1381/2787] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:00 2022] DELETE /identity/v3/projects/2043c59e8fc14ab39bf8c0bd02f1d10e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:00.933589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:00.934071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:00.934071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:00.956047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:00.957271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:01.018252 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1407/2788] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:01.026682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.026975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.027273 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.049026 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:01.050306 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:01.058764 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1382/2789] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:18:01.063223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.073702 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c8478-41fd-40c7-a794-26b50f34dd4f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.079319 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.079921 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.080198 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.080464 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.081137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:18:01.081508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.083598 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.085084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:01.097141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1408/2790] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:18:01.101717 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.111744 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128aba-d440-470b-9c39-914f3f826705 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.117217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.117806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.118082 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.118348 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.118974 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.120903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.158805 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1383/2791] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:01.163233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.174026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92dfb-dbdb-48b1-aa39-6b53eec3314a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.235633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.236296 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.236558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.236829 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.237480 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.239930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.262954 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1409/2792] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:01.267691 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.278825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9d870-a731-4ebf-9a36-79e6cf5c9cf7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.284408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.285090 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.285367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.285639 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.286334 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.288062 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.298665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1384/2793] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:01.303474 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.313985 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e24cc-ab32-44da-bd20-1b33e526ff75 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.319274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.319892 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.320165 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.320553 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.321193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.322782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e99ffb-dcef-4d11-b93a-92fe074469db admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.335993 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1410/2794] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:01.338960 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.349685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe8f21-6a84-464b-baae-8a6b2384e10f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.355877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.356538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.356830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.357203 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] PATH_INFO: `/v3/projects/27cfb22da25e4f1494e60c4dc87815f6/users/00f8c7d65f1d47acbec3d95ca34a94b0/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.372694 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cfb22da25e4f1494e60c4dc87815f6, user_id=00f8c7d65f1d47acbec3d95ca34a94b0, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.375389 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.390511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1385/2795] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:01 2022] PUT /identity/v3/projects/27cfb22da25e4f1494e60c4dc87815f6/users/00f8c7d65f1d47acbec3d95ca34a94b0/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:01.403289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.403821 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.404116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.433425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] MFA Rules not processed for user `00f8c7d65f1d47acbec3d95ca34a94b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:01.434646 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:01.503010 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1411/2796] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:01.507735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.570874 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96db4f-1872-4d41-95ff-d416b160ecd5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.575589 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.576210 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.576487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.576754 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.577396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.579410 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.608466 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1386/2797] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:01.613560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.625706 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3130f2c-371b-49fa-bdc4-1df2000d53d0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.685106 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.685713 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.685984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.686249 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.686872 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.688823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.697843 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:01.713795 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1412/2798] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:01.718258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.729806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dd26f-b836-4caf-94c3-78c3d28607b1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.735149 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.735765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.736052 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.736320 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.736989 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.738642 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.750240 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1387/2799] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:01.754482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.765273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d36cb-def0-492b-beb1-b7ee1f1c3a7e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.771294 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.771939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.772216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.772470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] PATH_INFO: `/v3/projects/a200fa39e8314e3f8b312bca1887ee32/users/d139c3cb5d804dbd89d56900b3e6798b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.788036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a200fa39e8314e3f8b312bca1887ee32, user_id=d139c3cb5d804dbd89d56900b3e6798b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.790672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.807273 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1413/2800] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:01 2022] PUT /identity/v3/projects/a200fa39e8314e3f8b312bca1887ee32/users/d139c3cb5d804dbd89d56900b3e6798b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:01.811301 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.822400 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486ad13-0164-4223-85fc-32aebe446e06 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.847992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:01.885507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.886108 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.886403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.886661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.887304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.888959 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f7b2f-a9a1-4882-9938-c7e9138f51d7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.899730 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1388/2801] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:01.904182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:01.915426 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba52fd-49fe-4774-9751-3fe1ddc61fb5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:01.920802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:01.921484 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.921766 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.922029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] PATH_INFO: `/v3/projects/a200fa39e8314e3f8b312bca1887ee32/users/d139c3cb5d804dbd89d56900b3e6798b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.924145 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a200fa39e8314e3f8b312bca1887ee32, user_id=d139c3cb5d804dbd89d56900b3e6798b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:01.927100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:01.937765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1414/2802] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:01 2022] PUT /identity/v3/projects/a200fa39e8314e3f8b312bca1887ee32/users/d139c3cb5d804dbd89d56900b3e6798b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:01.948472 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:01.948759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:01.949043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:01.977815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] MFA Rules not processed for user `d139c3cb5d804dbd89d56900b3e6798b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:01.979395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:02.063060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1389/2803] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:02.074984 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:02.076995 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:02.138496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a78d-cd48-46d4-a014-38e2809e90ff admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:02.142592 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:02.143242 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:02.143504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:02.143757 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:02.153405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:02.154831 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:02.164264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1415/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:02.249793 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:02.260649 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca6753-4517-4ed8-a8fd-fb23de3dfd0b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:02.264765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:02.265395 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:02.265669 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:02.265941 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:02.335115 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:02.336493 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:02.345927 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1390/2805] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:04.496405 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:04.498824 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:04.592584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8887ef-dac1-4fa9-83e6-061c64bd4810 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:04.599171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:04.600223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:04.600501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:04.600771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:04.603531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:04.620241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:04.622776 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677283b-5303-4ec5-b51f-4b47e3456960 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:04.636523 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1416/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:11.330722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:11.384119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:11.399943 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d48e-3a16-42fa-b42a-2781fd455dda service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:11.406213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:11.406897 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:11.407233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:11.407507 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] PATH_INFO: `/v3/users/00f8c7d65f1d47acbec3d95ca34a94b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:11.408809 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f8c7d65f1d47acbec3d95ca34a94b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:11.410750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:11.461317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f8c7d65f1d47acbec3d95ca34a94b0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:11.461877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f8c7d65f1d47acbec3d95ca34a94b0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:11.469096 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f8c7d65f1d47acbec3d95ca34a94b0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:11.480317 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f8c7d65f1d47acbec3d95ca34a94b0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:11.481036 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f8c7d65f1d47acbec3d95ca34a94b0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:11.500252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1391/2807] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:11 2022] DELETE /identity/v3/users/00f8c7d65f1d47acbec3d95ca34a94b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:11.505723 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:11.506213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:11.506312 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:11.528180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:11.529397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:11.574274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:11.600548 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1417/2808] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:11.611930 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:11.613685 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:11.678446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babe565-c9c1-4bb5-b591-4e688f7df4cf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:11.682447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:11.683059 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:11.683433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:11.683705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:11.693169 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:11.694594 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:11.704837 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1392/2809] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:12.035494 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:12.035855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:12.049588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b5107-b78f-488f-92e3-6051147b7a6f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:12.054815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:12.056122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.056402 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.056697 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] PATH_INFO: `/v3/projects/27cfb22da25e4f1494e60c4dc87815f6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:12.057968 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cfb22da25e4f1494e60c4dc87815f6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:12.060118 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:12.141709 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cfb22da25e4f1494e60c4dc87815f6', 'request_id': 'req-10dee0e6-c464-434a-9d63-be5aa7583016'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.148587 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cfb22da25e4f1494e60c4dc87815f6', 'request_id': 'req-10dee0e6-c464-434a-9d63-be5aa7583016'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.154854 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cfb22da25e4f1494e60c4dc87815f6', 'request_id': 'req-10dee0e6-c464-434a-9d63-be5aa7583016'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.171021 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] The token cache is being invalidate because project 27cfb22da25e4f1494e60c4dc87815f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:12.175750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.175750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.175750 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1418/2810] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:12 2022] DELETE /identity/v3/projects/27cfb22da25e4f1494e60c4dc87815f6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:12.177991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:12.188436 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39060500-bd08-41a2-9712-7670ec90718c service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:12.259252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:12.260375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.260375 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.260632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] PATH_INFO: `/v3/users/d139c3cb5d804dbd89d56900b3e6798b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:12.261881 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d139c3cb5d804dbd89d56900b3e6798b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:12.264093 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:12.276135 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:12.325397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd139c3cb5d804dbd89d56900b3e6798b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.326390 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd139c3cb5d804dbd89d56900b3e6798b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.334706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd139c3cb5d804dbd89d56900b3e6798b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.345152 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd139c3cb5d804dbd89d56900b3e6798b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.346179 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd139c3cb5d804dbd89d56900b3e6798b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.372443 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1393/2811] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:12 2022] DELETE /identity/v3/users/d139c3cb5d804dbd89d56900b3e6798b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:12.527137 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:12.530369 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:12.618967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dee0e6-c464-434a-9d63-be5aa7583016 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:12.696966 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:12.697610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.697893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.698161 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:12.700689 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:12.733545 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:12.735276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:12.745847 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:12.780674 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:12.782173 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cf0c3-b2d7-4f92-afed-c67a8eb09d8c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:12.792271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1419/2812] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:12.807676 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1c98-2c71-4de6-a1f9-04eeb41d81a4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:12.812680 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:12.813402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.813677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.814069 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] PATH_INFO: `/v3/projects/a200fa39e8314e3f8b312bca1887ee32` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:12.815229 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a200fa39e8314e3f8b312bca1887ee32)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:12.817151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:12.866114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a200fa39e8314e3f8b312bca1887ee32', 'request_id': 'req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.872611 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a200fa39e8314e3f8b312bca1887ee32', 'request_id': 'req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.878571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a200fa39e8314e3f8b312bca1887ee32', 'request_id': 'req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.893195 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] The token cache is being invalidate because project a200fa39e8314e3f8b312bca1887ee32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:12.893500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.894198 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3f68b921-3dad-4ce9-a488-dfb4b01bc1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:12.896260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1394/2813] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:12 2022] DELETE /identity/v3/projects/a200fa39e8314e3f8b312bca1887ee32 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:12.901110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.901470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.901741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:12.922074 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:12.923339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:12.989112 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1420/2814] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:12.998396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:12.998661 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:12.998935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.019511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:13.020737 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:13.029470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1395/2815] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:13.033970 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.044022 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9185431-c7c5-4eaa-804d-60724174213c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.049687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.050310 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.050629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.051677 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.052354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:18:13.052659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.054276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.067179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1421/2816] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:18:13.071722 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.081413 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335eeec-8ada-47fc-8f78-332896cad735 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.086297 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.086907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.087252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.087500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.088157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.090194 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.100165 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:13.122402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1396/2817] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:13.126774 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.137142 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05dda4-33fc-42fd-8360-235a9271a0a6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.195282 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.195918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.196220 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.196496 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.197121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.199234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.223371 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1422/2818] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:18:13.227401 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.238705 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d39fcb-e522-4d91-89ea-0c587427e4af admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.243861 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.244483 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.244766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.245038 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.245772 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.247454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.257935 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1397/2819] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:13.262208 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.273229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cc4a5-37f4-4958-b0a8-6cbcf393eb4b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.278989 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.279632 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.279910 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.280177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.280826 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.282483 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e725208-16cc-4140-a301-19332f74de25 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.298789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1423/2820] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:13.303477 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.313953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39ab9e-30b9-45cb-b6ca-f599226ae584 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.318950 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.319684 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.319959 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.320230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] PATH_INFO: `/v3/projects/878bb52f8c66491caa23b901225d438e/users/8988e23e14d64fc2a51941054a97d4b4/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.335450 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878bb52f8c66491caa23b901225d438e, user_id=8988e23e14d64fc2a51941054a97d4b4, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.338114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.356853 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1398/2821] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:13 2022] PUT /identity/v3/projects/878bb52f8c66491caa23b901225d438e/users/8988e23e14d64fc2a51941054a97d4b4/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:13.367530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.367814 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.368086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.396793 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] MFA Rules not processed for user `8988e23e14d64fc2a51941054a97d4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:13.398168 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:13.425732 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:13.471389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1424/2822] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:18:13.472821 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.538947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663866d-c0f2-4fe1-910b-de1b11f887e8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.544648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.545303 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.545585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.545855 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.547219 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.549238 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.580192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1399/2823] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:13.584504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.596172 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66475-b48f-4e6f-a920-5b492ec7ac70 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.657797 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.658407 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.658679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.658943 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.659605 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.661649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.687213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1425/2824] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:13.691433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.703015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b1159-a2fc-4523-834b-c67c1b599ad9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.708258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.708907 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.709187 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.709460 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.710154 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.712026 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.733207 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1400/2825] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:13.733933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.746419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dae34f-2ce8-4a8c-a583-906933390eee admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.751141 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.751835 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.752114 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.752383 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] PATH_INFO: `/v3/projects/c0aa718c2f3544a38137ed5063acc2fd/users/ca111855243d4d6296fb40d60b35b46a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.769930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0aa718c2f3544a38137ed5063acc2fd, user_id=ca111855243d4d6296fb40d60b35b46a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.772896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.790649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1426/2826] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:13 2022] PUT /identity/v3/projects/c0aa718c2f3544a38137ed5063acc2fd/users/ca111855243d4d6296fb40d60b35b46a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:13.795803 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.810449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae4f54-e828-421a-8b96-9676fa8c1def admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.880838 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.881471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.881766 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.882074 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.882741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.884486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86abbc-6288-4082-90d6-3254d824dd9f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.896213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1401/2827] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:13.900686 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:13.912501 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2ec56-108b-40ca-b2be-bb2478228c1a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:13.917536 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:13.918416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.918656 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.919012 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] PATH_INFO: `/v3/projects/c0aa718c2f3544a38137ed5063acc2fd/users/ca111855243d4d6296fb40d60b35b46a/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.921068 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0aa718c2f3544a38137ed5063acc2fd, user_id=ca111855243d4d6296fb40d60b35b46a, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:13.924096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:13.935615 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1427/2828] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:13 2022] PUT /identity/v3/projects/c0aa718c2f3544a38137ed5063acc2fd/users/ca111855243d4d6296fb40d60b35b46a/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:18:13.946597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:13.946871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:13.947192 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:13.979302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] MFA Rules not processed for user `ca111855243d4d6296fb40d60b35b46a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:13.980898 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:13.994638 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:14.057699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1402/2829] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:14.070607 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:14.073042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:14.134070 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b12364-b5e7-4b13-ad10-75fcafed911b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:14.138271 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:14.138975 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:14.139342 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:14.139676 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:14.150683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:14.152277 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:14.162179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1428/2830] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:14.245412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:14.256168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ea9ba-bbd1-43e1-a651-078412df71ca None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:14.259877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:14.260571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:14.261552 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:14.261902 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:14.327660 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:14.329016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:14.338272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1403/2831] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:19.425944 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:19.427439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:19.456216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:19.497151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102e970-5b51-48a1-b0ad-ed8f804342a9 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:19.502679 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:19.503341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:19.503612 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:19.503877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] PATH_INFO: `/v3/users/62f5d4659eea4d91a689d709460396d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:19.505019 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f5d4659eea4d91a689d709460396d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:19.506939 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:19.561036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f5d4659eea4d91a689d709460396d2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.573026 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f5d4659eea4d91a689d709460396d2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.580537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f5d4659eea4d91a689d709460396d2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.581357 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f5d4659eea4d91a689d709460396d2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.581978 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f5d4659eea4d91a689d709460396d2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.599918 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1429/2832] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:19 2022] DELETE /identity/v3/users/62f5d4659eea4d91a689d709460396d2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:19.789914 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:19.801806 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d56a8-8d2c-4278-bd94-cbd36454f043 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:19.859360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:19.859997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:19.860278 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:19.861158 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] PATH_INFO: `/v3/projects/e6db5b66c75c499d98d02d6e0041299f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:19.862304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6db5b66c75c499d98d02d6e0041299f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:19.864332 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:19.913095 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6db5b66c75c499d98d02d6e0041299f', 'request_id': 'req-96dda806-cde5-4247-b7a7-2093bf42b7cb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.914106 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:19.921449 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6db5b66c75c499d98d02d6e0041299f', 'request_id': 'req-96dda806-cde5-4247-b7a7-2093bf42b7cb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.927908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6db5b66c75c499d98d02d6e0041299f', 'request_id': 'req-96dda806-cde5-4247-b7a7-2093bf42b7cb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.944146 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] The token cache is being invalidate because project e6db5b66c75c499d98d02d6e0041299f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:19.944433 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.945065 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:19.946566 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1404/2833] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:19 2022] DELETE /identity/v3/projects/e6db5b66c75c499d98d02d6e0041299f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:19.951564 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:19.953688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:19.973362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e74a78-758c-4299-9e76-832c5a87280d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:20.036542 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:20.037171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:20.037411 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:20.037707 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] PATH_INFO: `/v3/users/a66f90f59af641da976dde35d8b4394b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:20.038860 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66f90f59af641da976dde35d8b4394b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:20.040811 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:20.088194 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:20.091233 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66f90f59af641da976dde35d8b4394b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.105047 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66f90f59af641da976dde35d8b4394b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.111519 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66f90f59af641da976dde35d8b4394b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.112179 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66f90f59af641da976dde35d8b4394b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.112787 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66f90f59af641da976dde35d8b4394b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.128214 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1430/2834] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:19 2022] DELETE /identity/v3/users/a66f90f59af641da976dde35d8b4394b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:20.323909 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:20.325512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:20.342050 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda806-cde5-4247-b7a7-2093bf42b7cb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:20.406362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:20.407008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:20.407322 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:20.407597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] PATH_INFO: `/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:20.408834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46652dc4c6e940c3a1dbb2c7dc057f3a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:20.410775 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:20.465001 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46652dc4c6e940c3a1dbb2c7dc057f3a', 'request_id': 'req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.471250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46652dc4c6e940c3a1dbb2c7dc057f3a', 'request_id': 'req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.477555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46652dc4c6e940c3a1dbb2c7dc057f3a', 'request_id': 'req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.493034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] The token cache is being invalidate because project 46652dc4c6e940c3a1dbb2c7dc057f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:20.493342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.493988 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5ade6e0c-aaf6-4c8f-bc85-c81aa3e41de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:20.495593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1405/2835] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:20 2022] DELETE /identity/v3/projects/46652dc4c6e940c3a1dbb2c7dc057f3a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:23.746093 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:23.749292 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:23.768933 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05f69d-e006-48b6-b58e-72cb17dbe866 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:23.829479 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:23.830035 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:23.830313 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:23.830582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] PATH_INFO: `/v3/users/8988e23e14d64fc2a51941054a97d4b4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:23.831748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8988e23e14d64fc2a51941054a97d4b4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:23.833669 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:23.885412 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8988e23e14d64fc2a51941054a97d4b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:23.898288 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8988e23e14d64fc2a51941054a97d4b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:23.905577 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8988e23e14d64fc2a51941054a97d4b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:23.906256 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8988e23e14d64fc2a51941054a97d4b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:23.906866 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8988e23e14d64fc2a51941054a97d4b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:23.922569 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1431/2836] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:23 2022] DELETE /identity/v3/users/8988e23e14d64fc2a51941054a97d4b4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:23.927367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:23.927900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:23.927900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:23.947162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:23.948358 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:24.011326 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1406/2837] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:24.022250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:24.023972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:24.069979 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:24.087688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d725e-5b68-47ac-a106-a2339ab545b7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:24.091477 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:24.092095 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:24.092364 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:24.092627 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:24.102048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:24.103503 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:24.113554 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1432/2838] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:24.413673 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:24.416539 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:24.429506 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7107c-a663-405d-ba07-fdd99ff4f437 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:24.434585 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:24.435298 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:24.435581 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:24.435844 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] PATH_INFO: `/v3/projects/878bb52f8c66491caa23b901225d438e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:24.437016 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878bb52f8c66491caa23b901225d438e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:24.438929 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:24.441497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:24.499499 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878bb52f8c66491caa23b901225d438e', 'request_id': 'req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.505921 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878bb52f8c66491caa23b901225d438e', 'request_id': 'req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.512129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878bb52f8c66491caa23b901225d438e', 'request_id': 'req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.527880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] The token cache is being invalidate because project 878bb52f8c66491caa23b901225d438e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:24.528250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.529075 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.530965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1407/2839] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:24 2022] DELETE /identity/v3/projects/878bb52f8c66491caa23b901225d438e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:24.535258 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:24.545103 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1534d-2593-4adb-a90f-97959e33ba39 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:24.602672 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:24.603423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:24.603602 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:24.604234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] PATH_INFO: `/v3/users/ca111855243d4d6296fb40d60b35b46a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:24.605457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca111855243d4d6296fb40d60b35b46a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:24.607498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:24.658429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca111855243d4d6296fb40d60b35b46a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.670187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca111855243d4d6296fb40d60b35b46a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.677213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca111855243d4d6296fb40d60b35b46a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.677898 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca111855243d4d6296fb40d60b35b46a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.678504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ecf4ef36-e8e7-4a21-b3f7-71a74dff7df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca111855243d4d6296fb40d60b35b46a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:24.696755 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1433/2840] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:24 2022] DELETE /identity/v3/users/ca111855243d4d6296fb40d60b35b46a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:24.990561 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:24.992042 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:25.005359 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b57712-9a8d-49e7-8db8-29e3c6cfbc37 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.064501 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.065130 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.065403 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.065670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] PATH_INFO: `/v3/projects/c0aa718c2f3544a38137ed5063acc2fd` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.066780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0aa718c2f3544a38137ed5063acc2fd)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.068765 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.250019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0aa718c2f3544a38137ed5063acc2fd', 'request_id': 'req-413120ce-1de4-4623-987d-5992b5343d89'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:25.256569 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0aa718c2f3544a38137ed5063acc2fd', 'request_id': 'req-413120ce-1de4-4623-987d-5992b5343d89'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:25.262783 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0aa718c2f3544a38137ed5063acc2fd', 'request_id': 'req-413120ce-1de4-4623-987d-5992b5343d89'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:25.280231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] The token cache is being invalidate because project c0aa718c2f3544a38137ed5063acc2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:25.280489 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:25.281173 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-413120ce-1de4-4623-987d-5992b5343d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:25.283019 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1408/2841] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:24 2022] DELETE /identity/v3/projects/c0aa718c2f3544a38137ed5063acc2fd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:25.287885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.288498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.288498 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.308729 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:25.310053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:25.371583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1434/2842] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:25.437111 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.437391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.437645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.457690 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:25.458888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:25.467712 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1409/2843] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:25.472091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.477096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:25.484128 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cda00-5723-4096-a7f2-530d2c31018a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.489786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.490425 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.490704 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.490973 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.491655 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:18:25.491956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.493556 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.506024 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1435/2844] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:18:25.509643 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.519289 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf4c5-c62d-45c5-8106-8eb8a5c46479 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.524302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.524906 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.525185 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.525481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.526147 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.528120 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.543481 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:25.559905 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1410/2845] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:25.564356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.574649 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f290f0b-9da3-4eee-a6bd-a68b1767a348 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.636285 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.636905 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.637180 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.637543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.638341 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.640261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.662714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1436/2846] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:25.667010 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.677559 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b809f-49e4-482d-9125-4b00606916a0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.683464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.684063 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.684342 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.684610 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.685362 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.686990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.697344 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1411/2847] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:25.701640 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.712001 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def5f83-aabf-47f1-8305-68149ad208ed admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.717217 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.717819 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.718091 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.718359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.718994 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.720635 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf45aa-bb94-460c-b8ee-f3c17ff9854e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.731773 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1437/2848] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:25.736500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.748190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43d35-e0ae-4eb8-89dc-70ee084736fd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:25.754209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:25.755101 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.755432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.755721 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] PATH_INFO: `/v3/projects/e5e02f34e91b48ce9778153781f6e4d0/users/b17581c498064442a5a4e95b337b889e/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.774452 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e02f34e91b48ce9778153781f6e4d0, user_id=b17581c498064442a5a4e95b337b889e, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:25.777223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:25.792755 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1412/2849] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:25 2022] PUT /identity/v3/projects/e5e02f34e91b48ce9778153781f6e4d0/users/b17581c498064442a5a4e95b337b889e/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:18:25.803170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:25.803460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:25.803733 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:25.847537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] MFA Rules not processed for user `b17581c498064442a5a4e95b337b889e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:25.849476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:25.920748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1438/2850] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:25.925420 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:25.995423 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5651be-4b86-4b1d-b1c4-da234e0e0298 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.000663 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.001402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.001686 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.001953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.002577 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.004637 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.034347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1413/2851] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:25 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:26.039744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.051478 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099df013-9b27-428d-8897-9ff2b0f03518 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.102401 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:26.110261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.110919 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.111261 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.111533 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.112195 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.114435 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.139680 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1439/2852] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:26 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:18:26.143901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.154876 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd781-8327-4635-a925-081181b82a4b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.159726 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.160406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.160682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.160946 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.161621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.163312 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.173234 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1414/2853] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:26.177700 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.188555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25932dc-2360-4289-948b-fadfda074ab9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.193470 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.194073 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.194351 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.194621 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.195305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.196908 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aebb-814b-4d67-884f-5a9df80d9845 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.207182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1440/2854] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:26.211926 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.224266 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe55c8-c952-4205-9b5f-173fd43dbe3e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.230576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.231271 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.231551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.231932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] PATH_INFO: `/v3/projects/85f0ee4004cc4f37ad15bc38b1f59886/users/b1e8c877262147899d420478f80220b8/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.248112 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f0ee4004cc4f37ad15bc38b1f59886, user_id=b1e8c877262147899d420478f80220b8, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.251299 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.266021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1415/2855] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:26 2022] PUT /identity/v3/projects/85f0ee4004cc4f37ad15bc38b1f59886/users/b1e8c877262147899d420478f80220b8/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:26.277583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.277870 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.278140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.307247 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] MFA Rules not processed for user `b1e8c877262147899d420478f80220b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:26.308508 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:26.386262 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1441/2856] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:26.391367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.454779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2a75e-761c-4858-8769-3adb08d22b9f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.459886 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.460485 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.460759 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.461024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.461671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.463633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.482580 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:26.498728 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1416/2857] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:26 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:18:26.504912 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.519584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545682f-754a-4b36-bf8d-0522acdefdfe None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.581146 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.581660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.581937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.582224 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.582865 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.584839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.607880 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1442/2858] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:26 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:18:26.612035 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.622997 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b64e0e-c621-4b44-94dd-6e0dfeb4ab6a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.629391 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.630044 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.630361 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.630614 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.631319 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.632947 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.643596 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1417/2859] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:26.649509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.664716 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c26af-64c8-40db-a552-b843ad05deb9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.672362 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.673014 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.673287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.673555 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] PATH_INFO: `/v3/projects/42a88e7f76b4492791f09e9106434aa9/users/06bd1bb2e2b64c2590c3e5a2e2c89be8/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.689537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a88e7f76b4492791f09e9106434aa9, user_id=06bd1bb2e2b64c2590c3e5a2e2c89be8, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.692231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.707568 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1443/2860] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:26 2022] PUT /identity/v3/projects/42a88e7f76b4492791f09e9106434aa9/users/06bd1bb2e2b64c2590c3e5a2e2c89be8/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:26.711709 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.725070 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cfc9e-e9b2-4b63-9cbe-430e1e0c5b0b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.789753 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.790555 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.790894 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.791333 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.792276 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.794786 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272de84f-f133-4702-af04-3850022ac69e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.810402 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1418/2861] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:26.815651 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.826701 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e66d9-8ef7-4ace-ba92-be948891ee4c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:26.832659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:26.833237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.833517 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.833780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] PATH_INFO: `/v3/projects/42a88e7f76b4492791f09e9106434aa9/users/06bd1bb2e2b64c2590c3e5a2e2c89be8/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.836023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a88e7f76b4492791f09e9106434aa9, user_id=06bd1bb2e2b64c2590c3e5a2e2c89be8, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:26.838660 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:26.850744 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1444/2862] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:26 2022] PUT /identity/v3/projects/42a88e7f76b4492791f09e9106434aa9/users/06bd1bb2e2b64c2590c3e5a2e2c89be8/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:26.863313 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:26.863797 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:26.864148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:26.896975 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] MFA Rules not processed for user `06bd1bb2e2b64c2590c3e5a2e2c89be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:26.898209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:26.968816 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1419/2863] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:26.981334 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:26.983338 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:27.005530 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:27.045204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53f5d-b494-4d0b-8c46-80fa2a690b92 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:27.049512 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:27.050190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:27.050471 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:27.050738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:27.060171 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:27.061610 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:27.071457 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1445/2864] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:27.611002 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:27.612715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:27.625459 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5292691-fa77-4163-8a8a-05e2ff1d68c7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:27.629627 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:27.630222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:27.630503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:27.630756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:27.692512 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:27.694005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:27.703354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1420/2865] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:31.823309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:31.833445 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6fb9-71e6-41e6-8cc8-c2e774300d4c service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:31.838032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:31.838620 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:31.838942 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:31.839260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:31.900274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:31.901738 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b14e1-945b-48f9-9fe0-42a66692a6d4 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:31.911343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1446/2866] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:34.091891 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:34.096435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:34.157958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422884b8-db4c-4486-8760-d8305b1fcfdf service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:34.164601 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:34.165522 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:34.165885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:34.166202 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] PATH_INFO: `/v3/users/b17581c498064442a5a4e95b337b889e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:34.168031 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17581c498064442a5a4e95b337b889e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:34.171337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:34.229958 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17581c498064442a5a4e95b337b889e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.230657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17581c498064442a5a4e95b337b889e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.238034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17581c498064442a5a4e95b337b889e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.251824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17581c498064442a5a4e95b337b889e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.252880 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17581c498064442a5a4e95b337b889e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.274086 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1421/2867] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:34 2022] DELETE /identity/v3/users/b17581c498064442a5a4e95b337b889e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:34.278802 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:34.279344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:34.279344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:34.300158 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:34.301431 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:34.370878 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1447/2868] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:34.382597 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:34.384068 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:34.445071 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493dea-1827-4f89-921c-e5978b3e3669 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:34.449796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:34.450398 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:34.450678 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:34.450932 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:34.461723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:34.463237 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:34.472796 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1422/2869] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:34.773076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:34.774714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:34.788823 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422018f-203a-4440-b0b8-29e7a74450cd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:34.793714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:34.794356 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:34.794638 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:34.794887 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] PATH_INFO: `/v3/projects/e5e02f34e91b48ce9778153781f6e4d0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:34.796061 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e02f34e91b48ce9778153781f6e4d0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:34.797959 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:34.849678 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e02f34e91b48ce9778153781f6e4d0', 'request_id': 'req-ab993888-8348-4822-9423-8a6ea34579a1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.856385 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e02f34e91b48ce9778153781f6e4d0', 'request_id': 'req-ab993888-8348-4822-9423-8a6ea34579a1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.857588 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:34.863315 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e02f34e91b48ce9778153781f6e4d0', 'request_id': 'req-ab993888-8348-4822-9423-8a6ea34579a1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.876661 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] The token cache is being invalidate because project e5e02f34e91b48ce9778153781f6e4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:34.876948 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.877566 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:34.879084 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1448/2870] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:34 2022] DELETE /identity/v3/projects/e5e02f34e91b48ce9778153781f6e4d0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:34.883465 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:34.893521 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb714-6cba-4c81-aa03-b1b34c9edbf6 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:34.915048 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:34.954670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:34.955325 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:34.955571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:34.955843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] PATH_INFO: `/v3/users/b1e8c877262147899d420478f80220b8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:34.956967 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e8c877262147899d420478f80220b8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:34.958990 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:35.006870 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e8c877262147899d420478f80220b8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.007482 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e8c877262147899d420478f80220b8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.014181 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e8c877262147899d420478f80220b8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.025656 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e8c877262147899d420478f80220b8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.026355 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e8c877262147899d420478f80220b8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.047260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1423/2871] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:34 2022] DELETE /identity/v3/users/b1e8c877262147899d420478f80220b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:35.335511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:35.336889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:35.351482 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab993888-8348-4822-9423-8a6ea34579a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:35.418962 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:35.420023 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:35.420748 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:35.421034 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] PATH_INFO: `/v3/projects/85f0ee4004cc4f37ad15bc38b1f59886` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:35.422213 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f0ee4004cc4f37ad15bc38b1f59886)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:35.424354 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:35.480896 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f0ee4004cc4f37ad15bc38b1f59886', 'request_id': 'req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.488589 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f0ee4004cc4f37ad15bc38b1f59886', 'request_id': 'req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.495243 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f0ee4004cc4f37ad15bc38b1f59886', 'request_id': 'req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.510064 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] The token cache is being invalidate because project 85f0ee4004cc4f37ad15bc38b1f59886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:35.510239 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.510889 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.512527 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1449/2872] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:35 2022] DELETE /identity/v3/projects/85f0ee4004cc4f37ad15bc38b1f59886 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:35.516616 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:35.518144 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:35.532053 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926df91-389b-4450-bb2a-417f9ea51d0e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:35.600461 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:35.601114 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:35.601386 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:35.601651 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] PATH_INFO: `/v3/users/06bd1bb2e2b64c2590c3e5a2e2c89be8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:35.602872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bd1bb2e2b64c2590c3e5a2e2c89be8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:35.605162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:35.654841 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:35.656889 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bd1bb2e2b64c2590c3e5a2e2c89be8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.657818 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bd1bb2e2b64c2590c3e5a2e2c89be8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.666872 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bd1bb2e2b64c2590c3e5a2e2c89be8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.675910 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bd1bb2e2b64c2590c3e5a2e2c89be8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.676511 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-73cb4e53-92d1-4773-9ee8-e161cb0f9ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bd1bb2e2b64c2590c3e5a2e2c89be8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:35.693294 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1424/2873] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:18:35 2022] DELETE /identity/v3/users/06bd1bb2e2b64c2590c3e5a2e2c89be8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 04:18:35.983075 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:35.984560 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:35.998335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf122eaf-3c3f-4522-a16d-f59abb8ecca4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.056972 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.057581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.057861 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.058138 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] PATH_INFO: `/v3/projects/42a88e7f76b4492791f09e9106434aa9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.059324 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a88e7f76b4492791f09e9106434aa9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.061516 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.077361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:36.108683 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a88e7f76b4492791f09e9106434aa9', 'request_id': 'req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:36.114828 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a88e7f76b4492791f09e9106434aa9', 'request_id': 'req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:36.121206 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a88e7f76b4492791f09e9106434aa9', 'request_id': 'req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:36.137591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] The token cache is being invalidate because project 42a88e7f76b4492791f09e9106434aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:18:36.137937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:36.138834 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f7ff4c3-34e9-4691-97bc-4b1a1c489d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:18:36.141150 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1450/2874] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:18:35 2022] DELETE /identity/v3/projects/42a88e7f76b4492791f09e9106434aa9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:36.149017 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.149368 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.149648 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.170464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:36.171885 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:36.251789 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1425/2875] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:36.260947 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.261240 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.261629 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.285031 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:36.286298 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:36.295708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1451/2876] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:36.300372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.311706 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11712cf1-ed6d-41f7-89c8-0a148ed5ea09 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.317447 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.318041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.318302 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.318591 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.319292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:18:36.319593 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.321193 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.333560 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1426/2877] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:18:36.338264 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.347929 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df7346-5865-4ea3-a9ff-fc38e1b8a4d9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.354888 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.355509 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.355789 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.356056 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.356687 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.358601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.390049 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1452/2878] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:18:36 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:36.394454 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.409671 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b7813-270d-4e89-bb89-92ff76bf504f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.469496 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.470091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.470365 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.470633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.471288 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.473213 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.499247 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1427/2879] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:18:36 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:18:36.503696 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.514930 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4e781-5463-47cb-94a1-35d7efc953ac admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.520408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.521015 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.521276 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.521771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.522465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.524096 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.534237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1453/2880] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:36.539300 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.556991 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0e45d-4f2d-4c0a-8f4a-c2e4457cb17e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.566310 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.567252 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.567632 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.568003 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.568971 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.570572 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee3f71-5b7d-4cac-91c1-6485225fa415 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.581606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1428/2881] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:18:36.586770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.602076 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56cdd3-3d1a-4e35-bc50-1fa3eb8b3630 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.608392 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.609046 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.609323 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.609593 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] PATH_INFO: `/v3/projects/e87b3289f518477683649369d5ddaaa0/users/97ccfdf0b23e49fb8b121ddfc05f6755/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.627886 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87b3289f518477683649369d5ddaaa0, user_id=97ccfdf0b23e49fb8b121ddfc05f6755, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.630608 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.648710 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1454/2882] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:18:36 2022] PUT /identity/v3/projects/e87b3289f518477683649369d5ddaaa0/users/97ccfdf0b23e49fb8b121ddfc05f6755/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:18:36.662845 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.663248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.663622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.695602 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] MFA Rules not processed for user `97ccfdf0b23e49fb8b121ddfc05f6755`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:18:36.696851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:36.716756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:36.769027 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1429/2883] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:18:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:18:36.784708 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:18:36.786765 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:18:36.873119 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592166-e126-4818-8ac5-17f34bac25d2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:18:36.876893 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77972e7e64f47c7a3fe7153841976c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:18:36.877504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:18:36.877782 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:18:36.878048 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:18:36.887647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:18:36.889378 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06afaae-d22e-4005-a076-1cd74e0a2665 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:18:36.898996 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1455/2884] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:18:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:18:46.766524 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=19 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:18:46.894327 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:07.418217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:07.484931 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0a612-8ef6-42f3-b593-5c61ad6f92c7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:07.490253 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:07.490877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:07.491188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:07.491470 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] PATH_INFO: `/v3/users/97ccfdf0b23e49fb8b121ddfc05f6755` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:07.492621 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ccfdf0b23e49fb8b121ddfc05f6755)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:07.494541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:07.550014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ccfdf0b23e49fb8b121ddfc05f6755'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:07.550859 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ccfdf0b23e49fb8b121ddfc05f6755'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:07.557851 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ccfdf0b23e49fb8b121ddfc05f6755'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:07.567568 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ccfdf0b23e49fb8b121ddfc05f6755'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:07.568476 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ccfdf0b23e49fb8b121ddfc05f6755'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:07.594633 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1430/2885] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:19:07 2022] DELETE /identity/v3/users/97ccfdf0b23e49fb8b121ddfc05f6755 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:07.600585 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:07.601187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:07.601187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:07.621190 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:07.622393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:07.686422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1456/2886] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:07.697230 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:07.698779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:07.759263 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3931-9d21-411f-807d-d4656d028c68 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:07.764008 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:07.764615 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:07.764888 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:07.765163 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:07.775949 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:07.777387 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4930aa-247c-4e7a-bd30-7f048b975059 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:07.788110 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1431/2887] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:08.092037 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.093595 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:08.107546 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b10cb-adee-4187-9699-10fead38bbb4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.114307 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.115071 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.115257 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.115544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] PATH_INFO: `/v3/projects/e87b3289f518477683649369d5ddaaa0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.117539 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87b3289f518477683649369d5ddaaa0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.119830 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.171728 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87b3289f518477683649369d5ddaaa0', 'request_id': 'req-25bd152e-f5c2-4606-82aa-b20675051ace'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:08.178305 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87b3289f518477683649369d5ddaaa0', 'request_id': 'req-25bd152e-f5c2-4606-82aa-b20675051ace'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:08.184234 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87b3289f518477683649369d5ddaaa0', 'request_id': 'req-25bd152e-f5c2-4606-82aa-b20675051ace'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:08.198967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] The token cache is being invalidate because project e87b3289f518477683649369d5ddaaa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:19:08.199281 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:08.199904 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-25bd152e-f5c2-4606-82aa-b20675051ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:08.201408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1457/2888] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:19:08 2022] DELETE /identity/v3/projects/e87b3289f518477683649369d5ddaaa0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:08.206699 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.207151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.207503 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.227756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:08.229500 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:08.250011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:08.294543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1432/2889] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:08.303604 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.303881 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.304149 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.324685 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:08.325875 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:08.334268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1458/2890] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:08.338630 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.350094 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b5970-d126-49fc-8987-a84d8b047762 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.356005 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.356604 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.356884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.357151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.357798 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:19:08.358104 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.359771 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.372248 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1433/2891] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 04:19:08.376652 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.386245 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842da58-0747-4b8a-8c0b-0a379cfbd1b4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.391511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.392108 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.392379 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.392641 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.393287 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.395204 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.425032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1459/2892] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:19:08 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:19:08.429432 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.439486 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de4a0e-d1a1-47de-91d0-d2add3245135 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.496834 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.497426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.497715 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.497963 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.498579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.500520 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.523107 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1434/2893] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:19:08 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:19:08.527352 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.538386 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b19e2-089d-45db-b654-59d9b4f3c998 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.544193 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.544798 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.545080 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.545361 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.546008 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.547647 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.557928 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1460/2894] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:19:08.562134 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.572372 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438959c5-d763-4324-97d9-b4bebe2121e2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.577434 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.578014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.578285 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.578538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.579233 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.580824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaaa840-579a-43da-8104-87df2deba4b6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.591407 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1435/2895] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 04:19:08.595450 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.606140 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984272-782a-403f-80b2-88eaeb18ae0e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.611659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.612260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.612537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.612809 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] PATH_INFO: `/v3/projects/04897b8010e84bfc9b15fc72edcea163/users/0ffcb3c156f646c1926b599f64c4e8ba/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.628290 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04897b8010e84bfc9b15fc72edcea163, user_id=0ffcb3c156f646c1926b599f64c4e8ba, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.631011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.647000 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1461/2896] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:19:08 2022] PUT /identity/v3/projects/04897b8010e84bfc9b15fc72edcea163/users/0ffcb3c156f646c1926b599f64c4e8ba/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:08.659579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.659982 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.660426 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.688779 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] MFA Rules not processed for user `0ffcb3c156f646c1926b599f64c4e8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:08.690000 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:08.758162 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1436/2897] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:08.771192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:08.773033 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:08.795659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:08.831419 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f36af-f996-49bd-a757-1efa6200fde9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:08.835724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a1ecf64ce544ee5aa7839acde57bea2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:08.836322 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:08.836597 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:08.836863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:08.846599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:08.848177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea0211-6ef8-4d41-a9ed-0770d44e6a17 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:08.857885 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1462/2898] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:18.754900 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:18.853403 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=17 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:25.321890 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:25.389137 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2571d5-a35e-40cb-9241-374fe8b1d45e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:25.394118 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:25.394735 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:25.395014 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:25.395311 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] PATH_INFO: `/v3/users/0ffcb3c156f646c1926b599f64c4e8ba` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:25.396468 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffcb3c156f646c1926b599f64c4e8ba)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:25.398412 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:25.447571 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffcb3c156f646c1926b599f64c4e8ba'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:25.448217 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffcb3c156f646c1926b599f64c4e8ba'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:25.455129 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffcb3c156f646c1926b599f64c4e8ba'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:25.464211 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffcb3c156f646c1926b599f64c4e8ba'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:25.464840 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffcb3c156f646c1926b599f64c4e8ba'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:25.482985 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1437/2899] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:19:25 2022] DELETE /identity/v3/users/0ffcb3c156f646c1926b599f64c4e8ba => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:25.487839 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:25.488339 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:25.488460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:25.507925 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:25.509131 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:25.577132 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1463/2900] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:25.587222 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:25.588677 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:25.651212 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907279d-96d7-44ce-8b7d-6e5ccba574fd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:25.654953 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:25.655590 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:25.655871 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:25.656143 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:25.665087 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:25.666538 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8b98-3156-4062-abef-e7fd071a88e7 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:25.676848 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1438/2901] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:25.953869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:25.955423 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:25.968221 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdee1b9-7ca2-40b2-9f96-cfa118bf3b63 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:25.973815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:25.974475 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:25.974754 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:25.975009 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] PATH_INFO: `/v3/projects/04897b8010e84bfc9b15fc72edcea163` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:25.976690 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04897b8010e84bfc9b15fc72edcea163)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:25.979045 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.031778 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04897b8010e84bfc9b15fc72edcea163', 'request_id': 'req-77726a6c-316d-4712-a10c-8c841517e824'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:26.040780 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04897b8010e84bfc9b15fc72edcea163', 'request_id': 'req-77726a6c-316d-4712-a10c-8c841517e824'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:26.047274 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04897b8010e84bfc9b15fc72edcea163', 'request_id': 'req-77726a6c-316d-4712-a10c-8c841517e824'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:26.062500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] The token cache is being invalidate because project 04897b8010e84bfc9b15fc72edcea163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:19:26.062791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:26.063446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-77726a6c-316d-4712-a10c-8c841517e824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:19:26.064956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1464/2902] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:19:25 2022] DELETE /identity/v3/projects/04897b8010e84bfc9b15fc72edcea163 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:26.070446 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.070732 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.071006 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.095682 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:26.096824 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:26.119836 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:26.172139 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1439/2903] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:26.180502 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.180786 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.181053 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.204211 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:26.205507 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:26.213864 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1465/2904] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:26.218272 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.228102 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b93976-fd4a-4aa4-85dd-146165e17778 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.233579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.234180 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.234455 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.234854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.235543 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 04:19:26.235850 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.237435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.249216 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1440/2905] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 31 04:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 04:19:26.253639 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.262855 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a665b8-6411-46a3-92fb-7ca3ddbe493b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.267877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.268500 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.268771 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.269011 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.269634 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.271583 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.302289 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1466/2906] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:19:26 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:19:26.307100 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.319497 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3b6e8-9fca-42b5-aa2e-98fedc5d355c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.378556 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.379241 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.379354 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.379606 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.380224 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.382148 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.414741 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1441/2907] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:19:26 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:19:26.419160 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.430181 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aece9c-c46f-40b5-b7c5-8ababddd5698 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.436465 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.437130 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.437397 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.437667 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.438309 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.439937 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.451520 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1467/2908] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:19:26.456088 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.466435 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b50f73-cafb-47dd-baa5-212273945202 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.471408 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.472034 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.472316 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.472578 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.473223 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.474825 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780af24-d462-40d3-ba65-85fb3777600c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.486131 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1442/2909] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:19:26.490659 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.501408 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175e79-8c3c-44f0-a4ce-fcf4ee03b3d2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.509499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.510462 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.510816 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.511155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] PATH_INFO: `/v3/projects/89d0e9ff07ea4aaea9087d1558cade5f/users/b2af8118d23a4eb184ff229ac0583e45/roles/27e940be03fb44c8974bdff2bc53de9b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.527511 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d0e9ff07ea4aaea9087d1558cade5f, user_id=b2af8118d23a4eb184ff229ac0583e45, role_id=27e940be03fb44c8974bdff2bc53de9b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.530241 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.546094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1468/2910] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:19:26 2022] PUT /identity/v3/projects/89d0e9ff07ea4aaea9087d1558cade5f/users/b2af8118d23a4eb184ff229ac0583e45/roles/27e940be03fb44c8974bdff2bc53de9b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:26.551383 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d5c30-c1ae-402b-9084-85644a11a59f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.551665 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d5c30-c1ae-402b-9084-85644a11a59f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.551933 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d5c30-c1ae-402b-9084-85644a11a59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.572021 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ca4d5c30-c1ae-402b-9084-85644a11a59f None None] MFA Rules not processed for user `2be81f96a297433a9f04b4dfdb3e8453`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:26.573780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ca4d5c30-c1ae-402b-9084-85644a11a59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:26.640105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1443/2911] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 04:19:26.650117 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:26.652086 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:26.674741 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:26.714355 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac4e8e-02fc-420d-bc9b-873675d12ccc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:26.718783 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:26.719439 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:26.719714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:26.719981 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:26.729584 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:26.731433 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:26.741857 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1469/2912] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:30.811553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:30.812385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:30.812385 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:30.847109 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] MFA Rules not processed for user `b2af8118d23a4eb184ff229ac0583e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:30.848553 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:30.869822 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:30.926473 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1444/2913] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:30.931237 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:30.941920 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d1e23-954d-45c4-9863-bdce186b6a1b service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:30.947791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:30.948416 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:30.948688 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:30.948952 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:30.949693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:30.951693 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:30.983229 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1470/2914] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 31 04:19:30 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 04:19:30.987531 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:30.998903 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199cf66-98b4-44c9-b658-42be5fb0aaa1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:31.060865 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:31.061487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:31.061487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:31.061487 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:31.062843 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:31.064812 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:31.092125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1445/2915] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 31 04:19:30 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 04:19:31.093644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:31.104899 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c61b0-f12e-4171-af65-2c55da808c2c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:31.110250 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:31.110877 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:31.111170 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:31.111448 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:31.112097 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:31.113714 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:31.125201 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1471/2916] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:19:31.129579 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:31.140338 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd6f04-e573-4f9e-b317-3579461d8427 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:31.144986 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:31.145647 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:31.145922 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:31.146190 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] PATH_INFO: `/v3/projects/1916da62ca0c45ceaa974bf1112d8abc/users/e242426416054b79a5aa438529fc5b3b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:31.162324 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1916da62ca0c45ceaa974bf1112d8abc, user_id=e242426416054b79a5aa438529fc5b3b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:31.165043 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:31.183215 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1446/2917] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:19:31 2022] PUT /identity/v3/projects/1916da62ca0c45ceaa974bf1112d8abc/users/e242426416054b79a5aa438529fc5b3b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:31.184558 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:31.196808 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53088c-6814-4f2d-aaeb-85253c5ed277 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:31.259122 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:31.259767 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:31.260043 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:31.260303 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:31.260938 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:31.262544 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32d72d-7201-4264-a204-2b0dce30c2ca admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:31.272756 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1472/2918] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 31 04:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 04:19:31.277089 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:31.288174 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d536-74e6-4b5c-a5fc-4faed0bdd706 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:31.293471 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:31.294157 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:31.294430 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:31.294698 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] PATH_INFO: `/v3/projects/1916da62ca0c45ceaa974bf1112d8abc/users/e242426416054b79a5aa438529fc5b3b/roles/3f9414c863e34276b7483d736ee6a5b0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:31.296955 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1916da62ca0c45ceaa974bf1112d8abc, user_id=e242426416054b79a5aa438529fc5b3b, role_id=3f9414c863e34276b7483d736ee6a5b0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:31.299668 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:31.310490 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1447/2919] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 31 04:19:31 2022] PUT /identity/v3/projects/1916da62ca0c45ceaa974bf1112d8abc/users/e242426416054b79a5aa438529fc5b3b/roles/3f9414c863e34276b7483d736ee6a5b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:19:35.788671 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095862c9-d050-45aa-b586-ca3b28d1c70f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:35.789563 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095862c9-d050-45aa-b586-ca3b28d1c70f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:35.789990 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095862c9-d050-45aa-b586-ca3b28d1c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:35.821191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-095862c9-d050-45aa-b586-ca3b28d1c70f None None] MFA Rules not processed for user `e242426416054b79a5aa438529fc5b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:35.823051 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-095862c9-d050-45aa-b586-ca3b28d1c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:35.911094 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1473/2920] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:35.937378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:35.939710 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:36.022908 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fecbf-8b71-4f24-b8af-3dc82004a206 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:36.029091 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:36.029711 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:36.030182 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:36.030622 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:36.037337 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:36.099815 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:36.102011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:36.113901 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1448/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:37.138956 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:37.139668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:37.139668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:37.160770 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] MFA Rules not processed for user `b2af8118d23a4eb184ff229ac0583e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 04:19:37.161988 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:37.169210 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1474/2922] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 31 04:19:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 04:19:46.108842 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=24 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:47.154359 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:19:52.834150 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:52.896504 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca84068-ae1e-4d85-91ee-c7f858e4f66a service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:52.901025 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:52.901613 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:52.901884 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:52.902159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:52.904188 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:52.915067 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:52.916533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:52.924523 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1449/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:19:53.418681 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:19:53.420268 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:53.491531 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6330e4-2b0e-48ab-bf94-8f09595b2e04 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:19:53.496570 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:19:53.497177 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:19:53.497454 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:19:53.497725 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:19:53.499476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:19:53.510815 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:19:53.512442 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:19:53.522136 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1475/2924] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:19:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:20:02.921367 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=26 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:03.518308 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=32 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:06.653675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:06.655268 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:06.673304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c0f4e-c789-49ae-bb0e-742aabbee0ef service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:06.679159 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:06.679795 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:06.680072 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:06.680347 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:06.682098 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:06.695250 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:06.696895 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:06.708854 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1450/2925] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:20:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:20:09.218363 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:09.220218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:09.233473 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c274d5a-bcb5-4312-8b55-d0213c047316 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:09.238859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:09.239769 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:09.240042 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:09.240335 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:09.250883 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:09.252537 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:09.262389 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1476/2926] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:20:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:20:16.703780 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:19.258216 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:20:38.436396 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:20:38.449257 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87d67-6fa0-4a05-8fc9-be0bde31f329 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:20:38.455011 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:20:38.455645 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:20:38.455924 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:20:38.456191 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:20:38.458105 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:20:38.469852 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:20:38.471484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:20:38.481675 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1451/2927] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:20:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:20:48.477231 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:05.591613 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:05.602344 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51b35f-c1fc-4543-b323-13ff37106612 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:05.607582 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:05.608711 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:05.609036 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:05.609348 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:05.611460 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:05.626987 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:05.628452 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:05.640859 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1477/2928] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:22:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:22:06.863445 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:06.873719 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785697-3e68-49d1-81e8-3f28f6cc8067 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:06.877938 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:06.878600 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:06.878877 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:06.879184 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:06.880992 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:06.896015 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:06.897464 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:06.909291 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1452/2929] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:22:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:22:15.635406 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:16.903670 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:19.143304 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.152358 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b826-ab8d-46a5-aa1f-f13c2ac545d6 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.156156 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.156760 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.157032 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.157300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.159223 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:19.169722 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.171446 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.180231 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1478/2930] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 04:22:19.288060 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:19.297518 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff257-92d0-40dd-abe7-77ee9dd41faf service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:19.301551 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:19.302260 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:19.302533 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:19.302829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:19.304998 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:19.318912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:19.320693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:19.331475 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1453/2931] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:22:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:22:29.176050 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:29.326756 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:22:45.134541 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:22:45.144673 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c771eeb-df0a-4168-a9c5-8ca7c716ade6 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:22:45.149199 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d87047620874e09bc5d851978949acc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:22:45.149791 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:22:45.150127 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:22:45.150393 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:22:45.152504 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:22:45.162967 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:22:45.164422 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:22:45.172724 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1479/2932] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:22:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:22:55.169116 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:39.894869 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:39.908168 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804adbf-7a00-484e-a985-0dbde4535264 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:39.916457 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:39.917037 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:39.917381 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:39.917727 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:39.919830 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:39.934119 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:39.937703 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:39.955292 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1454/2933] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:23:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:23:43.002644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:43.017396 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23845bb-8df5-43c9-a164-c56272887a26 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:43.023228 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed122ea299374f9cb2b8b4854df14d65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:43.023907 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:43.024219 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:43.024528 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:43.026510 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:43.040300 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:43.041895 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:43.053476 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1480/2934] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:23:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:23:48.042618 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:48.044723 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:48.062778 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66de9e-9e1c-489c-a81e-a73304de3755 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:48.068628 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:48.069498 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:48.069961 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:48.070527 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:48.072360 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:48.087657 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:48.090912 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:48.113024 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1455/2935] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:23:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:23:51.147601 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:51.215191 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de71139-e138-488b-9efb-eb6e9feb8250 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:51.220218 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:51.220844 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:51.221121 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:51.221388 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] PATH_INFO: `/v3/users/b2af8118d23a4eb184ff229ac0583e45` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:51.222540 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2af8118d23a4eb184ff229ac0583e45)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:51.224499 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:51.282029 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2af8118d23a4eb184ff229ac0583e45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.294644 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2af8118d23a4eb184ff229ac0583e45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.302562 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2af8118d23a4eb184ff229ac0583e45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.303429 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2af8118d23a4eb184ff229ac0583e45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.304182 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2af8118d23a4eb184ff229ac0583e45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.325187 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1481/2936] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:23:51 2022] DELETE /identity/v3/users/b2af8118d23a4eb184ff229ac0583e45 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:23:51.506125 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:51.519304 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fcd32-ddec-41a2-bacb-682313bc7e7b service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:51.578161 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:51.578787 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:51.579066 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:51.579411 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] PATH_INFO: `/v3/projects/89d0e9ff07ea4aaea9087d1558cade5f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:51.580462 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d0e9ff07ea4aaea9087d1558cade5f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:51.582378 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:51.631693 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d0e9ff07ea4aaea9087d1558cade5f', 'request_id': 'req-08eda622-a2d0-4652-826c-1450b857e946'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.637835 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d0e9ff07ea4aaea9087d1558cade5f', 'request_id': 'req-08eda622-a2d0-4652-826c-1450b857e946'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.645792 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d0e9ff07ea4aaea9087d1558cade5f', 'request_id': 'req-08eda622-a2d0-4652-826c-1450b857e946'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.664576 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] The token cache is being invalidate because project 89d0e9ff07ea4aaea9087d1558cade5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:23:51.664965 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.665951 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.668151 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1456/2937] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:23:51 2022] DELETE /identity/v3/projects/89d0e9ff07ea4aaea9087d1558cade5f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:23:51.673110 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:51.674599 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:51.687837 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb09fbe-68d9-4e7e-bf36-22aef616ebd5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:51.747891 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:51.748591 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:51.748869 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:51.749133 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] PATH_INFO: `/v3/users/e242426416054b79a5aa438529fc5b3b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:51.750302 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e242426416054b79a5aa438529fc5b3b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:51.752260 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:51.759100 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:51.801668 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e242426416054b79a5aa438529fc5b3b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.813325 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e242426416054b79a5aa438529fc5b3b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.820863 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e242426416054b79a5aa438529fc5b3b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.821581 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e242426416054b79a5aa438529fc5b3b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.822192 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e242426416054b79a5aa438529fc5b3b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:51.841343 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1482/2938] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 31 04:23:51 2022] DELETE /identity/v3/users/e242426416054b79a5aa438529fc5b3b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:23:52.015964 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:23:52.017893 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:23:52.031883 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda622-a2d0-4652-826c-1450b857e946 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:23:52.091541 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be81f96a297433a9f04b4dfdb3e8453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a624bee162274d80bf52fd9ee90c7807', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:23:52.092209 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:23:52.092484 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:23:52.092750 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] PATH_INFO: `/v3/projects/1916da62ca0c45ceaa974bf1112d8abc` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:23:52.093849 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1916da62ca0c45ceaa974bf1112d8abc)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:23:52.095802 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:23:52.102055 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:23:52.145258 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1916da62ca0c45ceaa974bf1112d8abc', 'request_id': 'req-3b52268e-b07b-42f9-84fc-4b765b416fa6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:52.151829 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1916da62ca0c45ceaa974bf1112d8abc', 'request_id': 'req-3b52268e-b07b-42f9-84fc-4b765b416fa6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:52.157514 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1916da62ca0c45ceaa974bf1112d8abc', 'request_id': 'req-3b52268e-b07b-42f9-84fc-4b765b416fa6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:52.174041 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] The token cache is being invalidate because project 1916da62ca0c45ceaa974bf1112d8abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 04:23:52.174397 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:52.175406 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3b52268e-b07b-42f9-84fc-4b765b416fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 04:23:52.177599 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1457/2939] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 31 04:23:52 2022] DELETE /identity/v3/projects/1916da62ca0c45ceaa974bf1112d8abc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 04:24:01.833992 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:24:02.169451 dv-cinder-853910-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 04:25:47.390913 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 04:25:47.392534 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:47.541013 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206f8b-c68b-4e71-b4dc-a6469a38e6d6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 04:25:47.603543 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd2e18144a794d36b3d81cd3c8ef9cc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d2fe1d7f6114b92a3075d8345d8a067', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 04:25:47.604151 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 04:25:47.604424 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 04:25:47.604691 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 04:25:47.606524 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 04:25:47.618155 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 04:25:47.619845 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6474f-ffaf-4afe-8b8c-e9309f20a997 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 04:25:47.629633 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1483/2940] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 31 04:25:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 04:25:57.625873 dv-cinder-853910-5-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}